From recent reading and discussions it seems that lots of people aren’t using the Espressif boot loader (or my own rBoot, but that’s less of a surprise) on the ESP8266. Why? Maybe people aren’t aware of the reasons why you might want to. Or maybe they can’t figure out how to – when I started playing with the boot loader it was poorly documented (probably still is) and I had to work it out myself.
I talk about “rom” or “roms” here to mean full compiled user apps, that might traditionally have been deployed on their own but with a boot loader you can have several on the flash with just one operating at a time. To avoid confusion (hopefully) I’ll refer to the rom section of code (the code that is run from rom, usually just the .irom0.text elf section) as irom from now on.
Why use a boot loader?
The main reason is to allow you to have multiple “roms” and to be able to switch between them. That may not sound quite as useful as being able to dual boot your computer between Windows and Linux, but there are uses for it. For example, if you want to update your device over the air (OTA) you’ll need to have at least two rom slots on your flash, a running one and one that’s getting flashed with the new version (which you then reboot into). There are work-arounds you could do to OTA update a device from the running rom, but it wouldn’t be very safe. OTA updates are probably the main reason for wanting to use a boot loader. However, you might have a need to deploy a device with two completely different functions and not want to combine them into a single application. With a boot loader you could put both separate apps on the device and switch between them remotely or with a GPIO etc.
Another reason is that the boot loader can load your application differently to the built in loader and add features not present in the original loader. For example the built in loader checks a checksum for the elf sections it loads into iram, but not for the .irom0.text section that is run from rom (this is often referred to as the SDK lib, but that’s also where your code goes if you mark it with ICACHE_FLASH_ATTR). A boot loader could add this extra check (rBoot can, it’s coded but not currently pushed to git), the Espressif loader doesn’t).
The Espressif 1.2 loader introduced a new format for the roms that puts the irom section (aka SDK lib) first and the iram sections immediately after. This means you don’t need to work with the arbitrary split of iram going before 0x40000 and irom after (you can change this split, but you may need to keep changing it as your sections change). Now the iram sections are still limited in size (because there is a finite amount of iram) but the irom section can be pretty much as large as the flash chip (minus the space needed for the iram sections, boot loader and SDK config (last 4 sectors of flash)), without needing to play with the linker scripts. With rBoot your options for laying out the flash are unlimited, so at present I’m not supplying sample linker files, but I do explain how to make them (it’s not hard).
How do you use the boot loader?
1) Think about how you want to lay out your flash, particularly how many roms you want and if they should be the same sizes. See below for a worked example.
2) Compile your code in the normal way.
3) Link your code slightly differently. For each rom slot on the flash you need a linker script – a copy of the standard eagle.app.v6.ld with one simple change. You need to link your object files against each one, to produce multiple elf files. See below for examples and an explanation.
4) Use esptool to build a rom image from each of the linked elf files, this time using the ‘boot_v1.2+’ option.
5) Write the boot loader to 0x00000 on the flash.
6) If you want something more than the simple 2 rom default create and write a boot loader config sector to the flash at 0x01000.
6) Write the roms to the flash at the appropriate addresses (see below).
Why do you need new linker files and why do you need to link several times? I’ll assume if you are programming in C you understand the concepts of compiling and linking (but if you don’t it doesn’t really matter). The boot loader copies most sections to iram, this means they will always end up where you want them to be in memory, regardless of where they get placed on the flash. However the irom sections (usually just .irom0.text) aren’t copied, access to that code is via the memory mapped SPI flash. The whole chip is mapped at a base address of 0x40200000 so when you have multiple roms on your flash the irom section for each rom will be at a different place on the flash and so a different place in memory. When the code is linked the linker needs to know where that code will be in memory and the way to tell it is via the linker script. Short version: each copy of the rom on the flash needs to have been linked differently depending on where it will be flashed.
I’ll make this one easy – a two rom setup, both the same size. A simple but common scenario (all you would need for an app with OTA updates) and rBoot will self configure for this when you run it.
We want two roms on the flash, so the sensible thing to do is place one at the beginning and the other half way along. We need to leave space for the boot loader and the config so we can’t put the first one at the very beginning, so we’ll start it at sector 3, flash address 0x02000. There is no reason that the second can’t start exactly half way into the rom, but for symmetry we’ll start that at half+0x02000 (and rBoot’s default config expects this). Lets say you have an ESP8266 board with an 4Mbit SPI flash, that second rom is going to be written to 0x42000, for 8MBit it will be 0x82000. If you have a flash larger than 8Mb the default position for the second rom will remain at 0x82000 due to the memory mapping limitation.
Now we need to make two appropriate linker files. Copy eagle.app.v6.ld to rom0.ld and rom1.ld. Edit rom0.ld and change the value of ‘irom0_0_seg org = ‘ from 0x40240000 to 0x40202010. This is the base memory mapped flash address (0x40200000) + our chosen flash address (0x02000) + the length of an extra header (0x10). Edit rom1.ld and set the value to 0x40282010. You can increase the len parameter in both of these too if you need more space for your irom section. Just don’t make it so big that it will overflow into the flash space of the next rom (or push the iram sections into it) i.e. the rom must come out at <512KB if you are going to fit two of them on a 1MB flash.
Now edit your Makefile and find the linker line, which is likely to start $(LD). You need to duplicate this and make one of them use rom0.ld, instead of eagle.app.v6.ld, and the other to use rom1.ld. Also make sure they output two different files, don’t have the second one write over the output of the first!
Now you should have two elf files, previously you would have just had one. What normally happens next appears to involve black magic, a Makefile, a shell script/batch file (gen_misc) and a python script (gen_appbin). This produces the flashable rom from the elf file. The whole thing is a mess and can be greatly simplified by using my esptool2 to build roms (other simplified tools also exist, I called mine esptool2 not to imply it is better or supersedes other tools, but just to distinguish it on my own system, way before I intended to release it). The key thing here is that you need to run it twice now, to produce two rom files from the two different elf files. You also need to instruct it to produce roms for what it describes as “boot_v1.2+” (the SDK boot loader v1.2+). However you are currently calling this will need to be updated, but I’d suggest switching to esptool2 if you’re using the original SDK code to do this.
Now just flash the three files (two roms and rBoot itself):
e.g. esptool.py –port COM7 -fs 8m 0x00000 rboot.bin 0x02000 rom0.bin 0x82000 rom1.bin
Using ‘-fs 8m’ here is important, it ensures the flash size is stored in the first few bytes of the flash, this will be read by rBoot to determine the flash size so it can work out where the half way point is.
On first boot you’ll see a message that a default config is being created and all being well the first rom will start. Assuming you got this far, hold on for part two where I’ll show you how to switch rom and/or OTA update from your app…
14 thoughts on “A boot loader tutorial for ESP8266 using rBoot”
I compiled rbot with SDK2.1.0 and as a result got this:
ets Jan 8 2013,rst cause:2, boot mode:(3,6)
load 0x40100000, len 1340, room 16
ho 0 tail 12 room 4
load 0x3ffe8000, len 660, room 12
rBoot v1.4.1 – firstname.lastname@example.org
Flash Size: 32 Mbit
Flash Mode: QIO
Flash Speed: 40 MHz
rBoot Option: Big flash
Booting rom 0.
Fatal exception (0):
epc1=0x40210dcf, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x0
Fatal exception (0):
epc1=0x40210dcf, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x0
Thanks for the wonderful write up. I managed to understand all of it, except the part … “+ the length of an extra header (0x10). Edit rom1.ld and set the value to 0x40282010. “.
What is the extra header (size 0x10) for? I could not find any reference to it in the ESP8266 Getting Started guide under the Flash section. As per the document, you have boot.bin (8kb in case of rboot) at 0x0 and then user1.bin immidiately after that. So user1.bin should be at 0x402020000.
But then when I looked at eagle.app.v6.new.1024.app1.ld, the section irom0_0_seg is indeed at 0x40202010.
This extra 16 byte header was added to the original rom format for use with the espressif bootloader (which rBoot was intended to be an open source, and improved, replacement for). You can see an explanation of how the original boot loader works here: https://richard.burtons.org/2015/05/17/decompiling-the-esp8266-boot-loader-v1-3b3/ including a description of this extra header.
Thanks. That explained everything !
Sorry I found the compilation error: the Basic_rBoot compiles also the boot loader and there was a missing ld file
I tried to compile the rBoot example Basic_rBoot
following instructions in the readme.txt file
The only change i made was to the Makefile-user.mk:
ESPTOOL2 ?= c:\Espressif\utils\esptool2.exe
but got the following error when building (on Windows a complete printout):
07:59:04 **** Build of configuration Sming for project Basic_rBoot ****
make -C /C/tools/sming/Sming/rboot
make: Entering directory `/C/tools/sming/Sming/rboot’
xtensa-lx106-elf-gcc -Os -O3 -Wpointer-arith -Wundef -Werror -Wl,-EL -fno-inline-functions -nostdlib -mlongcalls -mtext-section-literals -D__ets__ -DICACHE_FLASH -DBOOT_BIG_FLASH -c rboot-stage2a.c -o /c/tools/Sming/Basic_rBoot/out/build/rboot-stage2a.o
c:/espressif/xtensa-lx106-elf/bin/../lib/gcc/xtensa-lx106-elf/5.1.0/../../../../xtensa-lx106-elf/bin/ld.exe: cannot open linker script file c:/Espressif/ESP8266_SDK/ld/eagle.app.v6.ld: Invalid argument
collect2.exe: error: ld returned 1 exit status
make: *** [/c/tools/Sming/Basic_rBoot/out/build/rboot.elf] Error 1
make: Leaving directory `/C/tools/sming/Sming/rboot’
make: *** [out/firmware/rboot.bin] Error 2
07:59:05 Build Finished (took 969ms)
Please raise Sming issues on Github or Gitter.
“Use esptool to build a rom image from each of the linked elf files, this time using the ‘boot_v1.2+’ option.”
Do you mean esptool2?
I think I meant gen_appbin.py, but it’s so long since I actually used the standard sdk tools to create a rom I can’t be sure what I meant now! But using esptool2 is certainly the easiest option.
Yes, you’re right! The sdk method is via gen_misc.sh, but esptool2 is definitely easier.
Something I consider would be useful in the boot loader is the possibility to test the programs before writing it to flash. Since I am new to ESP8266 I don’t know if this is doable or not. I don’t know if every time I compile my program (I have not done it yet) I have to flash several binary files or just the one (Probably the first time I have to flash several but after that probably only one is enough). If only one must be flashed then I guess it would be doable to be able to send the program to ram and run it from there.
Another interesting feature would be to be able to load to RAM or Flash without pulling up/down some pins. I have been playing with Pogoplug 4 devices. When this device is reset it waits a couple of seconds for a pattern at the serial port. If that patten is received then device enters in a mode that allows to execute boot loaders commands. Among these commands are some for sending the program to ram and execute from there or even write it to flash (NAND) memory. Do you think it is doable?
I am not asking you to do any change to your programs. I am just figuring out some possibilities.
The problem with booting from memory is that the instruction ram is quite small and so the bulk of most programs resides on the flash, which is memory mapped at run time. This may not the the code you actually write yourself, but much of the SDK library code you link against is destined to stay on flash. If you don’t add anything to that you could flash it just once and write your program entirely in the ram segment, but otherwise you will need to update the flash part of the program too. I think esp_tool.py already provides the ability to do what you want by writing to memory (then jumping to the code) from flash mode, so no need to add it to rBoot. And you are correct, after you flash rBoot the first time (which is tiny so only takes a second anyway), there is only more more file to flash each time you build a new version of your app. Flashing over serial isn’t that quick, but if you want to speed things up use the OTA functionality to reflash it, that is much quicker.
To answer your second question, yes it would be possible to add a new write mode to rBoot to allow it to do flash updates over serial. This would make the code larger though and ideally you want to keep it as small as possible. Also I don’t see many advantages to adding this functionality, you wouldn’t use it for deployed devices only in development/ If you want to avoid the GPIO directly you can add a tiny circuit that uses the DTS line to handle the GPIO for you, esp_tool.py uses this and I never have to physically touch my boards when I’m flashing them, it resets them and puts them into flash mode automatically.
I thing I need to read more about the architecture to understand what is available, what not and finally know what can be done. Thank you.