Embedded Systems Design II Embedded Linux

From EdWiki

Building the Linux Kernel - Cross-compiling for a BeagleBone Black

Install GNU toolchain for the A-Profile Architecture

$ cd ~/Downloads
$ Download GNU Toolchain for the A-profile Architecture: 8.3-2019.03 from here
$ sudo tar xvf ~/Downloads/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C /opt/
$ export CC=/opt/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf/bin/arm-linux-gnueabihf-

Install Dependencies

$ sudo apt-get install bison flex lzop lzma libmpc-dev u-boot-tools libncurses5-dev:amd64 gettext
$ mkdir ~/bbb_build
$ cd ~/bbb_build

Test Cross Compiler

$  ${CC}gcc --version

arm-linux-gnueabihf-gcc (GNU Toolchain for the A-profile Architecture 8.3-2019.03 (arm-rel-8.36)) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO

Bootloader: U-Boot

Build U-Boot For BeagleBone

Das U-Boot – the Universal Boot Loader: U-Boot patch archive: u-boot-patches

$ cd ~/bbb_build
$ git clone
$ cd u-boot/
$ git checkout v2019.04 -b v2019.04
$ cd ~/bbb_build/u-boot
$ wget -c
$ wget -c
$ patch -p1 < 0001-am335x_evm-uEnv.txt-bootz-n-fixes.patch
$ patch -p1 < 0002-U-Boot-BeagleBone-Cape-Manager.patch
Configure and Build
$ make ARCH=arm CROSS_COMPILE=${CC} distclean
$ make ARCH=arm CROSS_COMPILE=${CC} am335x_evm_defconfig

Linux Kernel

This script will build the kernel, modules, device tree binaries and copy them to the deploy directory.

Mainline - Download

$ cd ~/bbb_build
$ git clone
$ cd bb-kernel/

For am33x-rt-v5.4 (Longterm 5.4.x + Real-Time Linux):

$ git checkout origin/am33x-rt-v5.4 -b am33x-rt-v5.4


$ ./


first run (to setup baseline tree): ./ then modify files under KERNEL directory then run (to rebuild with your changes): ./tools/

Root File System

Debian 9

User : debian
Password : temppwd


$ cd ~/bbb_build
$ wget -c


$ sha256sum debian-10.4-minimal-armhf-2020-05-10.tar.xz

sha256sum output

cd598e42850cbef87602bf15ee343abfbf0d8c6ba81028c741672b5f24263534  debian-10.4-minimal-armhf-2020-05-10.tar.xz


$ tar xf debian-10.4-minimal-armhf-2020-05-10.tar.xz

Setup microSD card

We need to access the External Drive to be utilized by the target device. Run lsblk to help figure out what linux device has been reserved for your External Drive.

$ lsblk

Dev sd card.jpg

Thus you would use:

$ export DISK=/dev/sdb 

Erase partition table/labels on microSD card:

$ sudo dd if=/dev/zero of=${DISK} bs=1M count=10

Install Bootloader

$ cd ~/bbb_build
$ sudo dd if=./u-boot/MLO of=${DISK} count=1 seek=1 bs=128k
$ sudo dd if=./u-boot/u-boot.img of=${DISK} count=2 seek=1 bs=384k

Create Partition Layout

$ sudo sfdisk ${DISK} <<-__EOF__

Format Partition

With mkfs.ext4 1.43, we need to make sure metadata_csum and 64bit are disabled. As the version of U-Boot needed for this target CAN NOT correctly handle reading files with these newer ext4 options.

$ sudo mkfs.ext4 -L rootfs -O ^metadata_csum,^64bit ${DISK}1

Mount Partition

On most systems these partitions may be auto-mounted...

$ sudo mkdir -p /media/rootfs/
$ sudo mount ${DISK}1 /media/rootfs/

Backup Bootloader

This version of MLO/u-boot.img will be used on the "eMMC" flasher script on this page.

$ cd ~/bbb_build
$ sudo mkdir -p /media/rootfs/opt/backup/uboot/
$ sudo cp -v ./u-boot/MLO /media/rootfs/opt/backup/uboot/
$ sudo cp -v ./u-boot/u-boot.img /media/rootfs/opt/backup/uboot/

Install Kernel and Root File System

Copy and paste that "export kernel_version=4.X.Y-Z" exactly as shown in your own build/desktop environment and hit enter to create an environment variable to be used later.

$ export kernel_version=5.4.70-bone-rt-r38

Copy Root File System

$ ~/bbb_build
$ sudo tar xfvp ./debian-10.4-minimal-armhf-2020-05-10/armhf-rootfs-debian-buster.tar -C /media/rootfs/
$ sync

Set uname_r in /boot/uEnv.txt

$ cd ~/bbb_build
$ sudo sh -c "echo 'uname_r=${kernel_version}' >> /media/rootfs/boot/uEnv.txt"

Copy Kernel Image

$ cd ~/bbb_build
$ sudo cp -v ./bb-kernel/deploy/${kernel_version}.zImage /media/rootfs/boot/vmlinuz-${kernel_version}

Copy Kernel Device Tree Binaries

$ cd ~/bbb_build
$ sudo mkdir -p /media/rootfs/boot/dtbs/${kernel_version}/
$ sudo tar xfv ./bb-kernel/deploy/${kernel_version}-dtbs.tar.gz -C /media/rootfs/boot/dtbs/${kernel_version}/

Copy Kernel Modules

$ cd ~/bbb_build
$ sudo tar xfv ./bb-kernel/deploy/${kernel_version}-modules.tar.gz -C /media/rootfs/

File Systems Table (/etc/fstab)

$ sudo sh -c "echo '/dev/mmcblk0p1 / auto errors=remount-ro 0 1' >> /media/rootfs/etc/fstab"


Edit: /etc/network/interfaces

$ sudo nano /media/rootfs/etc/network/interfaces


auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

Networking: Using a shared SD card with Multiple BeagleBone

To always enable the Ethernet interface as eth0.
Edit: /etc/udev/rules.d/70-persistent-net.rules

$ sudo nano /media/rootfs/etc/udev/rules.d/70-persistent-net.rules

# BeagleBone: net device ()
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

Remove microSD/SD card

$ sudo chown root:root /media/rootfs/
$ sudo chmod 755 /media/rootfs/
$ sync
$ sudo umount /media/rootfs

Boot new image

  • Connect the Micro SD Card to the BBB SD Card Connector
  • Power-Up the BBB ( You can use mini USB (P4) )
  • Connect Serial Cable to Beagleboard as explained here
  • Press and Hold S2 Button then press and release S3 Button

Note: To boot from the microSD card , you need to hold the button S2, this button is near the host USB port, on the other side of the ethernet port. If this is not held, the board may follow its default boot order and try booting from the onboard eMMC, if a uboot image is available on the onboard eMMC, our uboot image will not be built.

Flashing eMMC

To set up the standalone microSD image to automatically flash the eMMC on powerup. Login as debian (password = temppwd) and edit /boot/uEnv.txt with your preferred editor.

In /boot/uEnv.txt:

##enable BBB: eMMC Flasher:

Change to:

##enable BBB: eMMC Flasher:

Building and Deploying BeagleBone Black Kernel

mkdir ~/bbb_linux
cd ~/bbb_linux
alias armmake='make -j4 ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabihf-'

  1. Install ARM Linux toolchain
  2. Build U-Boot
  3. Build Linux Kernel for BeagleBone Black
  4. Build Root File System for BeagleBone Black
  5. Deploying the kernel
  6. Setup serial console

Getting started with Buildroot

$ mkdir ~/bbb_buildroot
$ cd ~/bbb_buildroot

Install ARM Toolchain

$ wget -c
$ sudo tar xf ./gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C /opt/

To use xconfig

$ sudo apt-get install libqt4-dev pkg-config

To use gconfig

$ sudo apt-get install gir1.2-gtk-2.0 libglade2-dev libcanberra-gtk-module

Initial configuration and build

Our first step to get started with Buildroot is obviously to build a minimal Linux system, with just a bootloader, Linux kernel and simple user-space.

The Buildroot at buildroot provides a ready-to-use defconfig for the BeagleBone Black platform. However, for educational purposes, we are going to start straight from the official Buildroot, and build our configuration from beaglebone_defconfig.

Getting Buildroot

Start by cloning Buildroot’s official Git repository:

$ git clone
$ cd buildroot/
$ git checkout 2019.08 -b 2019.08
$ make beaglebone_defconfig

Modifying default configuration

Now it’s time to start modifying our configuration:

$ make menuconfig

In the configuration, we’ll have to customize a number of options, as detailed below. Of course, take this opportunity to navigate in all the options, and discover what Buildroot can do.

  • In Toolchain
    Change Toolchain type to External toolchain. By default, Buildroot builds its own toolchain, but it can also use pre-built external toolchains. We’ll use the latter, in order to save build time. Since we’ve selected an ARM platform, a ARM toolchain is automatically selected, which will work for us.
Toolchain --->
    Toolchain type (External toolchain)  --->
       (X) External toolchain
    *** Toolchain External Options ***  
         (X) Arm ARM 2019.03
    Toolchain origin (Pre-installed toolchain)  --->
       (X) Pre-installed toolchain
       Toolchain path 
Buildroot external toolchain.png
  • In System configuration
    In System configuration, you can customize the System host name, System banner and root password if you wish. Keep the default values for the rest.
  • In Kernel
Kernel  --->
    (X) Custom Git repository
    URL of custom repository 
    Custom repository version
    Kernel configuration
        (X) Use the architecture default configuration
Buildroot custom git kernal.png

Keep the default values for the rest.

Running the build

To start the build, you can run just make.

$ make

The build will take a while (about 20-45 minutes). At the end of the build, the output is located in output/images. We have:

  • MLO, the first stage bootloader
  • u-boot.img, the second stage bootloader
  • zImage, the Linux kernel image
  • am335x-boneblack.dtb, the Linux kernel Device Tree Blob
  • rootfs.ext4, the root filesystem image, and
  • sdcard.img, ready for flashing.

Setup microSD card

Plug in your removable flash drive and run the lsblk command to identify the device. Here is the output of the 'lsblk' command on my system where ‘sdb’ is the removable flash storage:

sdb    8:16    1   7.4G  0  disk 
├─sdb1 8:17    1   16M   0  part /media/jshankar/4F75-58E3
└─sdb2 8:18    1   512M  0  part /media/jshankar/dfbb4848-2a5a-4d0c-85c2-4575ae64daa5

There are many command line tools to do the job, we will use parted utility for this tutorial. Run the parted command with the name of the block device that you want to format. In this case, it’s sdb. (Be careful with the name of the block device because you might end up formatting the wrong drive.)

Exchange sdb with the name of your block device in the following command:

$ sudo parted /dev/sdb

It will ask you to enter the password for the user and you will notice that parted replaces the username and $ sign, which means you are running the parted utility. First, let’s create a partition table. In this case, we are using MBR:

(parted) mklabel msdos

Once the partition table is created, you can create partitions on the drive. We will be creating just one partition:

(parted) mkpart primary ext2 1MiB 100%

Then set the boot flag on it:

(parted) set 1 boot on 

Exit the parted tool:

(parted) quit

Now format it as ext2:

sudo mkfs.ext2 /dev/sdb1

Copy Kernel Image

$ sudo dd if=./output/images/sdcard.img of=/dev/sdb bs=1M


$ sudo mount /dev/sdb2 /mnt

Edit: /mnt/etc/network/interfaces

$ sudo nano /mnt/etc/network/interfaces

# Configure Loopback
auto lo
iface lo inet loopback

#automatically start eth0, with dhcp
auto eth0
iface eth0 inet dhcp

Remove microSD/SD card

$ sync
$ sudo umount /mnt

Insert the SD card in the BeagleBone board. Press and Hold the Boot button (S2) and power it up. Provided you have a serial port connection at 115200 bps, you should see the system booting.

There is still plenty of work to do in our embedded system, but you’ve got the process uncovered.

The Hello World Linux Loadable Kernel Module

A kernel module is not an application — for a start there is no main() function! Some of the key differences are that kernel modules:

  • do not execute sequentially— a kernel module registers itself to handle requests using its initialization function, which runs and then terminates. The type of requests that it can handle are defined within the module code. This is quite similar to the event-driven programming model that is commonly utilized in graphical-user interface (GUI) applications.
  • do not have automatic cleanup — any resources that are allocated to the module must be manually released when the module is unloaded, or they may be unavailable until a system reboots.
  • do not have printf() functions — kernel code cannot access libraries of code that is written for the Linux user space. The kernel module lives and runs in kernel space, which has its own memory address space. The interface between kernel space and user space is clearly defined and controlled. We do however have a printk() function that can output information, which can be viewed from within user space.
  • can be interrupted — one conceptually difficult aspect of kernel modules is that they can be used by several different programs/processes at the same time. We have to carefully construct our modules so that they have a consistent and valid behavior when they are interrupted. The BeagleBone has a single-core processor (for the moment) but we still have to consider the impact of multiple processes accessing the module simultaneously.
  • have a higher level of execution privilege — typically, more CPU cycles are allocated to kernel modules than to user-space programs. This sounds like an advantage, however, you have to be very careful that your module does not adversely affect the overall performance of your system.
  • do not have floating-point support — it is kernel code that uses traps to transition from integer to floating-point mode for your user space applications. However, it is very difficult to perform these traps in kernel space. The alternative is to manually save and restore floating point operations — a task that is best avoided and left to your user-space code.


  1. /**
  2.  * file    hello.c
  3.  * author  J. Shankarappa
  4.  * date    4 Nov 2019
  5.  * version 0.1
  6.  * An introductory "Hello World!" loadable kernel module (LKM) that can display a message
  7.  * in the /var/log/kern.log file when the module is loaded and removed. The module can accept an
  8.  * argument when it is loaded -- the name, which appears in the kernel log files.
  9. */
  11. #include <linux/init.h>             /* Macros used to mark up functions e.g., __init __exit */
  12. #include <linux/module.h>           /* Core header for loading LKMs into the kernel */
  13. #include <linux/kernel.h>           /* Contains types, macros, functions for the kernel */
  15. MODULE_LICENSE("GPL");              /* The license type -- this affects runtime behavior */
  16. MODULE_AUTHOR("J.Shankarappa");     /* The author -- visible when you use modinfo */
  17. MODULE_DESCRIPTION("A simple Linux driver for the BBB.");  /* The description -- see modinfo */
  18. MODULE_VERSION("0.1");              /* The version of the module */
  20. static char *name = "World";        /* An example LKM argument -- default value is "world" */
  21. module_param(name, charp, S_IRUGO); /* Param desc. charp = char ptr, S_IRUGO can be read/not changed */
  22. MODULE_PARM_DESC(name, "The name to display in /var/log/kern.log");  /* parameter description */
  24. /** The LKM initialization function
  25.  *  The static keyword restricts the visibility of the function to within this C file. The __init
  26.  *  macro means that for a built-in driver (not a LKM) the function is only used at initialization
  27.  *  time and that it can be discarded and its memory freed up after that point.
  28.  *  
  29.  * returns : 0 if successful
  30.  */
  31. static int __init helloBBB_init(void){
  32.    printk(KERN_INFO "EmSys: Hello %s from the BBB LKM!\n", name);
  33.    return 0;
  34. }
  36. /** The LKM cleanup function
  37.  *  Similar to the initialization function, it is static. The __exit macro notifies that if this
  38.  *  code is used for a built-in driver (not a LKM) that this function is not required.
  39.  */
  40. static void __exit helloBBB_exit(void){
  41.    printk(KERN_INFO "EmSys: Goodbye %s from the BBB LKM!\n", name);
  42. }
  44. /** A module must use the module_init() and module_exit() macros from linux/init.h, which
  45.  *  identify the initialization function at insertion time and the cleanup function (as
  46.  *  listed above)
  47.  */
  48. module_init(helloBBB_init);
  49. module_exit(helloBBB_exit);

  • Line 15: The statement MODULE_LICENSE("GPL") provides information (via modinfo) about the licensing terms of the module that you have developed, thus allowing users of your LKM to ensure that they are using free software. Since the kernel is released under the GPL, your license choice impacts upon the way that the kernel treats your module.
  • Line 20: The name (ptr to char) is declared as static and is initialized to contain the string “hello”. You should avoid using global variables in kernel modules — it is even more important than in application programming, as global variables are shared kernel wide. You should use the static keyword to restrict a variable’s scope to within the module. If you must use a global variable, add a prefix that is unique to the module that you are writing.
  • Line 21: The module_param(name, type, permissions) macro has three parameters: name (the parameter name displayed to the user and the variable name in the module), type (the type of the parameter — i.e., one of byte, int, uint, long, ulong, short, ushort, bool, an inverse Boolean invbool, or a char pointer charp), and permissions (this is the access permissions to the the parameter when using sysfs. A value of 0 disables the entry, but S_IRUGO allows read access for user/group/others — See the Mode Bits for Access Permissions Guide)
  • Line 30 and 39: The functions can have whatever names you like (e.g., helloBBB_init() and helloBBB_exit()), however, the same names must be passed to the special macros module_init() and module_exit() on lines 48 and 49.
  • Line 32: The printk() is very similar in usage to the printf() function that you should be familiar with, and you can call it from anywhere within the kernel module code. The only significant difference is that you should specify a log level when you call the function. The log levels are defined in linux/kern_levels.h as one of KERN_EMERG, KERN_ALERT, KERN_CRIT, KERN_ERR, KERN_WARNING, KERN_NOTICE, KERN_INFO, KERN_DEBUG, and KERN_DEFAULT. This header is included via the linux/kernel.h header file, which includes it via linux/printk.h.

Essentially, when this module is loaded the helloBBB_init() function will execute, and when the module is unloaded the helloBBB_exit() function will execute.

Building the Module Code

A Makefile is required to build the kernel module — in fact, it is a special kbuild Makefile. The kbuild Makefile required to build the kernel module is shown below:


  1. MODULES := hello.o
  3. #guest architecture
  4. ARCH := arm
  6. CROSS_COMPILE := arm-linux-gnueabihf-
  7. obj-m := $(MODULES)
  9. # path of the arm compiled kernel
  10. KDIR := /path/to/compiled/linux/kernel
  11. DESTDIR := /path/to/compiled/kernel/target
  15. all: modules
  16. modules:
  17. 	$(MAKEARCH) -C $(KDIR) M=${shell pwd} modules
  19. install:
  20. 	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(DESTDIR) modules_install
  22. clean:
  23. 	$(MAKEARCH) -C $(KDIR) M=${shell pwd} clean

  • Line 7 of this Makefile is called a goal definition and it defines the module to be built (hello.o). obj-m defines a loadable module goal, whereas obj-y indicates a built-in object goal.
  • Line 17: The -C option switches the directory to the kernel directory before performing any make tasks. The M=${shell pwd} variable assignment tells the make command where the actual project files exist.
  • The modules target is the default target for external kernel modules. An alternative target is modules_install which would install the module (the make command would have to be executed with superuser permissions and the module installation path is required).
$ make

Testing the LKM

Embedded Linux Assignments

Assignment 1. Linux build

Build an embedded system Linux from the source code.
Steps involved in this exercise:

  1. Download U-boot source code and build MLO and U-boot images.
    1. Download kernel source code.
    2. Configure with the default configuration.
    3. Build kernel image and modules.
    4. If there the above steps are combined in a shell script, break them down and execute the steps separately.
  2. (execute one step at a time)
    1. Download a prebuilt rootfs.
    2. Download a source distribution.
    3. Build minimalistic rootfs using busybox.
  3. Place all these on an SD card and boot the system.
  4. (try this): NFS mount your project folder on the target, so that you can execute applications directly from the host computer without having to copy on the SD card.
  5. Write a hello world application and execute.
  6. Trace all the executables that execute till your application is executed. Do so by printing a greeting message in the following programs: U-boot, vmlinux kernel, init, bash and finally your application.

Assignment 2. System call

  • Enhance Linux kernel to add the following two system calls: sys_hello and sys_led, and write an application demonstrating its use.
  • sys_hello: prints a greetings message.
  • sys_led: turns on/off one of the LEDs on the board. The argument should specify OFF, ON or TOGGLE. If there are multiple LEDs, you may provide LED number as the second argument.

List of topics for presentation

Here is a list of possible topics. Some topics are more involved than others - use your discretion to decide the amount of information to cover in the specified time. (Each presentation may be 20-30 minutes; we will decide the exact time during the next class.)

1. Android architecture

Focus should be on how a Linux system is enhanced to make it an android system and not so much about feature details of android. The presentations should address aspects like changes in the kernel (if any), restructure of rootfs, layered architecture (native / Java services and application layer).

2. Multiprocessor support in Linux

Talk may address the following aspects: booting up, scheduling, interrupt distribution, handling shared memory.

3. Power management

Power modes, placing processor, memory and IO devices into various power states. This should be explained with an example implementation with one or two device drivers.

4. Virtualization

What is a virtual machine? What hardware/software support is required? How is VM implemented?

5. Realtime scheduler

What makes Linux non-realtime? How does the realtime scheduler address these limitations? Provide example(s) of applications that can be implemented with the realtime scheduler.

6. Version history of Linux

How has the Linux kernel evolved over the years? What are significant landmark versions?

7. Distributions

What are various distributions for embedded systems? (Angstrom, yocto, debian etc.) Provide details of one example distribution.

8. Other topics

Here are some additional topics which you may want to explore and suggest the content:

  1. Secure boot, kernel security
  2. Fuse: file system in user space
  3. sysfs file system
  4. Device tree blob