Users of rBoot have reported OTA problems when using Espressif SDK v1.5.1 which I have been able to reproduce on this version and v1.5.2. The problem appears to be in the sdk flash write functions. While most writes work fine some will occasional fails to fully write (leaving small areas of blank data (0xff)). To make matters worse no error is reported from the write function and attempts to read the flash back for verification immediately afterwards return the correct data (presumably they are serviced by the rom cache). This is very similar to the problem seen when trying to perform an OTA with insufficient power. I once had a similar report from a user using just a 100mA power supply and the problem was fixed by using a proper power supply. This makes me wonder if the new SDK causes the device to draw more power, although supplying extra power does not fix the problem in this case.
I have created a test case that doesn’t use rBoot but simply downloads a file and writes it to flash (which is of course very similar to what rBoot does) to demonstrate the problem to Espressif. I have yet to hear back from them. If you encounter this problem please add your support to my bug report. I have no reason to believe this bug will be limited to rBoot users and I assume SDK bootloader users and anyone else writing to the flash from an application (possibly only in long sustained writes) will also have this problem.
This is a good opportunity to encourage rBoot users to enable the irom checksum option, to help detect badly flashed roms at boot time. See the readme for more information.