Viciouss' brain.txt

Can I has Android now pls?

2023-06-20T21:00:00+00:00

Next Steps

There is a kernel image which might or might not boot, it has the device tree attached at the end of it, the inner workings of the boot image and the bootloader are no longer a mystery. Sounds like everything is in place to assemble the boot image, put it on the device and boot right into Android, right? Well, I have to tell you something.

Android has a history of heavily modifying the mainline kernel, introducing new and changing existing code. When the p4note device family was still new and shiny, there were hundreds of patches on top of mainline. This modified kernel is called the Android common kernel. These kernels were then picked up by the vendors who put another couple of hundred patches on top to get a device up and running. The end result were kernel trees that went up to hundreds of thousands of lines of code away from mainline. Upgrading from an old downstream kernel to mainline is therefor guaranteed to fail miserably. The system and vendor userspace parts (like the Android HAL and vendor binaries) expect certain files to exist, like sysfs and devfs entries, as well as certain uevents to be triggered. All this is missing.

Building Android from scratch is quite a huge task as well. It’s not like throwing in a kernel and then building it, the hardware abstraction layer will kick your butt if you are missing a required implementation for memory management, graphics and other things. A lot of configuration needs to be done before an image even comes out at the end of the build process. This is putting way too much complexity on top and with drivers missing, it will be an awful user experience anyway.

Ok then, how about starting small? There are a couple of options to chose from. The simplest solution is compiling a little init program to print out a hello world and packaging it with the kernel. But what to do with it next?

Maybe grabbing a copy of busybox and assembling a small system with a shell sounds great? For sure it does, but the goal is to write more driver code and not setting up an embedded system from scratch with half the drivers not working. So how about something more “usable” with tools already available?

Luckily, there are projects that provide small base systems with lots of customization options. They will provide a wide range of libraries and programs to be included in a linux system. Those projects also have build systems in place that will pick up all the tedious work and throw out a boot and system image at the end. Let’s have a look at one of them.

Introducing Buildroot

I chose buildroot for no apparent reason, I liked the name and it looked easy to get started with, so I just went with it.

So what is buildroot? It’s basically a collection of Make files, package descriptions and patches that, combined with a config file, will assemble a custom root filesystem tailored to your specific needs. There are hundreds of software packages to chose from and it uses the Linux kernel kconfig system for configuration. Sounds great for mainline development!

Buildroot in 10 Minutes

First, let’s download a copy of Buildroot. Anything not too old will be fine for this example. For the p4note project (see the end of this post), at least version 2023.02 is required for the 6.1 mainline kernel that is used.

To work with buildroot, there needs to be a board description for the target. This is either located in buildroot itself if it is officially maintained or in a so called external tree in a separate directory/repository. The board description usually contains config files and an overlay to the root file system for files that will be packaged along with it. A small skeleton project can be assembled in as little as three files:

Config.in
external.mk
external.desc

A defconfig file describes the selected options in buildroot. It is not mandatory, only the above files are, but for reasons you will see in a minute, it makes sense to already create it:

configs/p4note_defconfig

Only the external.desc file actually needs content to get started, the other files can be empty. It holds a name and a description for the board. For the p4note, I chose the following:

name: P4NOTE
desc: The Samsung Galaxy Note 10.1

The name should be A-Z0-9 all caps, as it will be part of the kconfig entries. Lowercase letters are technically allowed, but who would do such cruel thing? The description can be freely chosen, but it should fit in one line as it will be displayed in the menuconfig.

With this created, the configuration part can begin. In the buildroot directory, this command will copy the defconfig from the external tree to buildroot:

make BR2_EXTERNAL=/path/to/external/tree p4note_defconfig

It also sets up the external path as current context. All the following commands can be executed without the BR2_EXTERNAL. The next step is setting the configuration values:

make menuconfig

There are quite some options to chose from, I will describe the important ones in the next section. Afterwards, use F8 to save to the current file (which is .config in the buildroot dir) and quit.

The changes to the configuration are currently only in the buildroot directory, to finally apply these configuration changes to the external tree, do this:

make savedefconfig

This will update the defconfig in the configs directory of the external tree, as the current setup has the recommended directory structure and that’s how buildroot rolls.

As a last step, of course, a call to make will build the project.

Important Configuration

Buildroot uses a lot of sane defaults. It will use the latest available kernel for the buildroot release, recommended build utils, standard libaries and more. Of course there might be differences depending on what you need for your board, but for the p4note I only had to adjust the following:

Target options
- all the sub options
System configuration
- root password
Kernel -> Linux Kernel
- defconfig name, binary format for appended DTB and DTB name
Filesystem images
- an ext4 root file system to put the selected packages on

Have a look around, it’s not too scary and you should find your way through the menu. After this is done, you can go wild in the Target Packages menu, there are a ton of options in there. It’s possible to add custom packages in the external tree, have custom pre and post build scripts and more.

The Final Product

If you navigate to https://github.com/Viciouss/buildroot_external_p4note, there is a repository which contains a working configuration with wifi, bluetooth and touch screen already prepared. Wifi and Bluetooth is definitely working for the 3G version (N8000) and should also work for the other two variants, but I haven’t tested this.

Don’t forget to update wpa_supplicant.conf before building, this way it will connect to your wifi and automatically print the IP on the screen after boot so that you can easily SSH into the device. It will also print the USB ethernet IP as well if your prefer to connect this way. Make sure to change the USB IP of the host machine accordingly.

Resources

the buildroot documentation is very good and provides lots of useful information

The Boot Image

2022-01-24T18:00:00+00:00

Get to know your Boot Image

To boot the device with S-Boot as boot loader, you need to provide an image that has a special format. This format is described on the Boot Image Header page of the AOSP project. In our case, as the device was released with Android 4, this is the legacy header or version 0. It contains the following entries:

kernel image size and address
ramdisk size and address
second bootloader size and address
some meta data

The interesting part here is the kernel image and ramdisk, on the Galaxy Note 10.1 the second stage boot loader (S-Boot) is not part of the boot image. It resides on its own partition.

Creating a Boot Image

Fortunately, there is a tool to extract and assemble boot images for Android. It’s called abootimg and can be installed with the package manager on probably any of the popular linux distros. The man page should provide enough information to understand how it works.

What is not explained in detail is the boot image configuration file format. What does it look like? As I mentioned, you can extract boot.img files, so for your device this would be the way to go. Looking at the cfg file for the Note 10.1 and the AOSP description of the header format, you will probably see where this goes:

bootsize = 8388608
pagesize = 0x800
kerneladdr = 0x40008000
ramdiskaddr = 0x42000000
secondaddr = 0x40f00000
tagsaddr = 0x40000100
name = 

With this information, a sample call to create a boot.img for my case looks like this:

abootimg --create myboot.img \
        -f bootimg.cfg
        -r ramdisk.xyz
        -k zImage-dtb

Bootloader and the Boot Image

Now that you know how the image looks like, let’s have a look at the boot process, especially how the boot image is taken care off.

The second stage boot loader first looks at the boot partition and the boot image header. Depending on the numbers you provided, it puts everything in the right place in the memory of your device and then calls the kernel image entry point. As the device is from 2012 and neither uses nor supports a Device Tree entry in the boot image, parameters are still fed to the kernel with the ATAGS mechanism - that’s what the tagsaddr attribute is for.

The ATAGS or kernel tagged list is an area in RAM where the boot loader puts parameters for the kernel to consider. There is a hand full of parameters, from memory size and location, ramdisk size and location to serial number, revision and command line. The bootloader then puts the address of this list into the R2 register, next to the R1 register which contains the machine type id. Only with those two pieces of information, the kernel can chose the right path of what to do next.

Technically, you don’t need the R1 and R2 registers and the parameters from the kernel tagged list, as the Device Tree describes all necessary parameters for a specific device. Remember the config option to attach the DTB file at the end of the kernel? That’s what will work for the mainline kernel. Nevertheless, it is interesting to have these parameters from the boot loader to build the device tree in the first place.

Let’s have a look at what they are for the Note 10.1 and how the vendor boot image deals with them:

ATAG_CORE: 5 54410001 0 0 0
ATAG_MEM: 4 54410002 20000000 40000000
ATAG_MEM: 4 54410002 20000000 60000000
ATAG_MEM: 4 54410002 20000000 80000000
ATAG_MEM: 4 54410002 1FC00000 A0000000
ATAG_SERIAL: 4 54410006 aaaaaaaa aaaaaaaa
ATAG_INITRD2: 4 54420005 42000000 3a50bc
ATAG_REVISION: 3 54410007 6
ATAG_CMDLINE: ad 54410009 'console=ttySAC2,115200 loglevel=4 bootmode=2 sec_debug.level=0 sec_watchdog.sec_pet=5 androidboot.debug_level=0x4f4c sec_log=0x200000@0x46000000 sec_tima_log=0x200000@0x46202000 sec_avc_log=0x40000@0x46404000 s3cfb.bootloaderfb=0x5ec00000 sysscope=0xee000000 lcdtype=1 consoleblank=0 lpj=3981312 vmalloc=256m oops=panic pmic_info=1 cordon=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa connie=GT-N8000_OPEN_EUR_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa androidboot.emmc_checksum=3 androidboot.odin_download=1 androidboot.bootloader=N8000XXSDQA7 androidboot.selinux=enforcing androidboot.warranty_bit=0 androidboot.sec_atd.tty=/dev/ttySAC2 androidboot.serialno=aaaaaaaaaaaaaaaa snd_soc_core.pmdown_time=1000'
ATAG_NONE: 0 0

ATAG_MEM - describes the ram in four banks of 512 MiB each. The last one is actually 4 MiB short, my guess is that this is related to the bootloader process.

ATAG_SERIAL - the serial number of the device.

ATAG_INITRD2 - is set to the memory position of the ramdisk that was written to the boot.img configuration with abootimg.

ATAG_REVISION - holds a revision number which is used in several places in the vendor kernel to distinguish some board details.

ATAG_CMDLINE - again rather obvious.

ATAG_NONE - the end of the tagged list.

About the revision, this comes to play in GPIO configuration, sensor positions or even which touch screen IC is installed. Although both ot the ICs are actually enabled in the defconfig, I’ve never come across one of the international versions of this device that haven’t had the Atmel IC. The Synaptics touch screen might have been in a prototype version or maybe one of the many variants of the South Korean market which I haven’t been able to spot on any hardware info sites yet.

After evaluating all those parameters and finishing loading the kernel, the initrd logic will be executed on the vendor kernel. The ramdisk will be mounted at root and then the kernel runs /sbin/init. As this is already Android territory, we are done here.

As usual, the kernel docs have some more on booting ARM, e.g. the ARM boot documentation.

Resources

the original vendor sources I requested from the Samsung Open Source page.

Building the kernel

2021-07-17T16:30:00+00:00

The sobering

With the serial connection working, I could get my hands dirty building kernels, right? Well there, not so fast. Where to actually start on a 5.2 kernel? Aside from all the vendor changes and Android kernel patches, there have been 43 major updates in between the Samsung kernel and mainline with lots of internal API changes.Knowing only the old vendor kernel and how it is booted with a board file, I had to learn what a device tree is. Very disappointing at first, as there was this big hurdle before even being able to build the kernel, but after a while it clicked and as it turned out, I could very much benefit from this right in the next step.

Device Trees

Both the board file and the device tree describe the hardware of the device. The board file is explicit in this regard as it is written in C code and it loads the drivers directly, it’s rather inflexible and if you want to share code with similar devices, it can get messy quickly. The “modern” approach for ARM devices is a device tree. It’s a language and data structure on its own, it is abstract and can be split up in smaller parts. You have a common SoC for a dozen devices? Just include the SoC device tree in your board tree and add the additional drivers on top, you are good to go. The abstract nature of device tree files also enables them to be used in other places than the kernel, e.g. a bootloader like U-Boot.

For the linux kernel, device tree files are compiled into device tree binaries (dtb files) which can then be read on boot to set up the system accordingly. I could gather a lot of information from the existing Exynos4 dts files which powered some Samsung devices in mainline at that time. I added a new device tree, included the Exynos4412 one and looked for matching nodes in the midas tree, changed IRQ and GPIO configurations and then added some other nodes like the max17042 battery monitor for example. With this minimal tree in place, only the defconfig was missing to create an initial kernel image.

defconfig and building the kernel

A defconfig file describes which architecture code and drivers get compiled in the kernel image. The exynos_defconfig looked promising, it already contains all the Exynos4 base drivers and some more, so I added the missing drivers and build the kernel:

# compile for a different architecture, the kernel build needs to know this
export ARCH=arm
# a cross compiler toolchain is needed to do this of course
export CROSS_COMPILE=/opt/gcc-linaro-arm-linux-gnueabihf/bin/arm-linux-gnueabihf- 
# this command takes the existing exynos_defconfig and copies it as .config to the 
# out directory which is then used later on to build the kernel
make O=out exynos_defconfig
# before building, edit the .config file, there are some drivers needed for the note 10.1
# start the kernel config GUI, I prefer nconfig, there is others, use 'make help' for more
make O=out nconfig
# everything is in place, build the kernel, the -j parameters tells the compiler
# how many threads to use, nproc --all returns the number of processors in the system
make O=out -j$(nproc --all) 

The build usually only takes a couple of minutes on current hardware, from the result there are two files of interest:

out/arch/arm/boot/zImage
out/arch/arm/boot/dts/exynos4412-p4note-n8010.dtb

The first file is the kernel image itself with all the architecture code and drivers that have been selected. It may contain drivers that are not needed, as is the case for the exynos_defconfig. This defconfig file compiles a kernel that can be used across all the existing exynos devices. But then, how does the kernel know which drivers to load? That’s where the dtb file comes in. But now there is a problem. The version of S-BOOT installed on the Exynos4 devices only knows about a kernel image that includes the board init code, it doesn’t know how to handle a device tree. Luckily, there is a kernel option for this fallback case: CONFIG_ARM_APPENDED_DTB.

With this enabled, the kernel will look for the DTB file immediately after the kernel code itself. This means that assembling the kernel image with attached DTB file is as easy as:

cat out/arch/arm/boot/zImage out/arch/arm/boot/dts/exynos4412-p4note-n8010.dtb > zImage-dtb

Conclusion

Going from board file to device tree was an interesting learning experience and it makes sharing device configurations among similar devices an easy process without littering your code with #ifdefs.

Resources

the training material from bootlin in general
- in this case the Device Tree 101 presentation from Thomas Petazzoni
Tutorial: Building the Simplest Possible Linux System from Rob Landley

Starting the Samsung Galaxy Note 10.1 Journey

2020-11-18T16:00:00+00:00

When I began working on the Note 10.1 at the end of 2018, I thought it can’t be that difficult to patch the device up. Add some lines here, fix a compiler issue there. I’m a software developer after all, how difficult can kernel and Android development be? Oh boy was I wrong.

I started from not even knowing how to compile a kernel. Luckily, the XDA community has a large forum dedicated to topics like this. I found a tutorial and got going.

Fast forward some weeks and a lot of tutorials later, I was going through the old 3.0.x sources from the Note 10.1. The kernel being 7 years old at the time with all the code Samsung introduced, more #ifdefs than I can count and a lot of unusual approaches to solve their user space needs, the learning curve was a roller coaster. From This looks like fun! to You know nothing Jon Snow in mere seconds. It was quite demotivating at times.

Fast forward again, I somehow stumbled across forkbombs blog (archive.org) - and from there the goal was clear. There was no way around getting the Note 10.1 into mainline. The first thing I’d need to start would be a serial connection, unfortunately the Note 10.1 has a 30pin proprietary connector, but it is shared across a hand full of devices. For example on the Galaxy Tab - my day was saved.

After some amateur soldering and wiring things up to a CP2102 module, I got my first lines on the serial connection:

Samsung S-Boot 4.0 for GT-N8000 (Jan  5 2017 - 16:48:45)

EXYNOS4412(EVT 1.1) / 2044MB / 0MB / Rev 6 / N8000XXSDQA7 /(PKG_ID 0xc071018)

BOOTLOADER VERSION : N8000XXSDQA7

BUCK1OUT(vdd_mif) = 0x05
BUCK3DVS1(vdd_int) = 0x20
cardtype: 0x00000007
SB_MMC_HS_52MHZ_1_8V_3V_IO
mmc->card_caps: 0x00000311
mmc->host_caps: 0x00000311
[mmc] capacity = 30777344