I was a little hasty in my judgement that this problem could not be solved. While it still appears to be true that only 1MB can be mapped at a time, it is possible to choose which 1MB is mapped. How the mapping was performed was a mystery, to me at least, and I can’t find any info about it on the internet. It was obvious that it was performed by the SDK code, but I hadn’t worked out where.

Since I released rBoot, Espressif have released a new SDK with a new version of the boot loader. This allows you to have two 1MB roms, which is clearly working around the limitation. The nice people at Espressif obviously know the internals of the hardware and have documentation for it, so they can do things that the rest of us wouldn’t know how to (or even know if it was possible).

How does the SDK memory map the flash?

Decompiling the new version of the SDK and comparing it to an older version made it easier to find where the magic happens. The function is Cache_Read_Enable (not well named!) and does not appear to be documented anywhere on the internet. I’ve decompiled it and so I know what the function does, but it communicates with other hardware through memory mapped I/O. Without documentation for that hardware it not easy to really know what it going on beyond this function. As a result some trial and error was required.

The SDK uses new flash size options in the flash header to indicate flash layout as well as size. This method is limited to what the SDK supports and isn’t in a place you want to be rewriting when the config changes (e.g. on an OTA update). So how can rBoot replicate, and improve on, this functionality? Cache_Read_Enable is called from several places in the SDK, because the flash has to be unmapped before normal SPI reads and writes can take place. The SDK SPI read, write and various other functions handle this unmapping (and remapping afterwards) for you. These functions in older versions of the SDK called Cache_Read_Enable directly, but now they all call a wrapper method called Cache_Read_Enable_New, which handles the extra logic involved with rom selection. This gives us a single point which, if we can replace it, would allow us to control the mapping ourselves.

Replacing Cache_Read_Enable_New

So how do we replace it? I first tried using the gcc -wrap option, but it didn’t work. Most references to Cache_Read_Enable_New where replaced with my own code, except those in user_init. It seems that -wrap doesn’t work well when the function you are wrapping is called within the same compilation unit (.o file). Another option is to just define a new method of the same name to override the original. Normally having two matching methods would cause an error at link time, to avoid this we mark the original as ‘weak’ to allow it to be overridden. This isn’t quite as neat, because it requires a small modification to the original libmain.a, but it works!

So, after writing a suitable replacement for Cache_Read_Enable_New I have a working solution. It’s a pity we still can’t map more than 8Mbit at a time, but at least we can now use the whole of larger flash chips in chunks. The new code is now on GitHub. See the readme file for explanation of how to use big flash support.