Memory map limitation – workaround

I was a little hasty in my judgement that this problem could not be solved. While it still appears to be true that only 1MB can be mapped at a time, it is possible to choose which 1MB is mapped. How the mapping was performed was a mystery, to me at least, and I can’t find any info about it on the internet. It was obvious that it was performed by the SDK code, but I hadn’t worked out where.

Since I released rBoot, Espressif have released a new SDK with a new version of the boot loader. This allows you to have two 1MB roms, which is clearly working around the limitation. The nice people at Espressif obviously know the internals of the hardware and have documentation for it, so they can do things that the rest of us wouldn’t know how to (or even know if it was possible).

How does the SDK memory map the flash?

Decompiling the new version of the SDK and comparing it to an older version made it easier to find where the magic happens. The function is Cache_Read_Enable (not well named!) and does not appear to be documented anywhere on the internet. I’ve decompiled it and so I know what the function does, but it communicates with other hardware through memory mapped I/O. Without documentation for that hardware it not easy to really know what it going on beyond this function. As a result some trial and error was required.

The SDK uses new flash size options in the flash header to indicate flash layout as well as size. This method is limited to what the SDK supports and isn’t in a place you want to be rewriting when the config changes (e.g. on an OTA update). So how can rBoot replicate, and improve on, this functionality? Cache_Read_Enable is called from several places in the SDK, because the flash has to be unmapped before normal SPI reads and writes can take place. The SDK SPI read, write and various other functions handle this unmapping (and remapping afterwards) for you. These functions in older versions of the SDK called Cache_Read_Enable directly, but now they all call a wrapper method called Cache_Read_Enable_New, which handles the extra logic involved with rom selection. This gives us a single point which, if we can replace it, would allow us to control the mapping ourselves.

Replacing Cache_Read_Enable_New

So how do we replace it? I first tried using the gcc -wrap option, but it didn’t work. Most references to Cache_Read_Enable_New where replaced with my own code, except those in user_init. It seems that -wrap doesn’t work well when the function you are wrapping is called within the same compilation unit (.o file). Another option is to just define a new method of the same name to override the original. Normally having two matching methods would cause an error at link time, to avoid this we mark the original as ‘weak’ to allow it to be overridden. This isn’t quite as neat, because it requires a small modification to the original libmain.a, but it works!

So, after writing a suitable replacement for Cache_Read_Enable_New I have a working solution. It’s a pity we still can’t map more than 8Mbit at a time, but at least we can now use the whole of larger flash chips in chunks. The new code is now on GitHub. See the readme file for explanation of how to use big flash support.

4 thoughts on “Memory map limitation – workaround”

  1. Hello, Richard.

    rBoot is really impressive, we are using in our project (https://github.com/cesanta/smart.js).

    But we want to use the same firmware ( (i.e. linked with the same LD-script) for all slots.
    So, we decided to try Cache_Read_Enable_New. And went into troubles.

    It works (for us), if both FW are equal (at least have the same size) or differences are very small (ex: changed some string, w/out changing size).
    But then I try to change one of FW – it doesn’t boot.
    I double-checked everything I did, seems all is ok.

    In another words:
    1. We have FW#1 -> flash it & boot (via serial)
    2. We have FW#2 -> flash it via OTA
    2.1 if FW#1 = FW#2 (or, as I wrote, almost equal) – everything is fine, i change rboot config and it loads FW#2
    2.2 But, if FW#2 is modified version of FW#1 (added new code or something else) – rboot cannot load FW#2 (crashes on very early stages)

    Have you ever seen something like this before?

    PS: we use SDK 1.5.0, might be it a problem?

    1. Sounds like the roms aren’t linked correctly, or Cache_Read_Enable_New isn’t being correctly included in the rom. If big flash is enabled and you’ve just got the one rom flashed to two separate 1mb chunks, but Cache_Read_Enable_New isn’t working properly your second rom will still be getting most of its code from the first rom. Try this: erase the flash, flash only rBoot and the second rom, see what happens. It should fail to boot if this is the problem, but rBoot and only first rom will work fine. Drop me an email and I’ll help you work through it.

Leave a Reply