I have several Friendly Elec NanoPi boards. Recently I have been working on my NanoPi R2S Plus because Armbian didn't have support for it and while it booted the Armbian images for NanoPi R2S, the WAN port and eMMC didn't work.
I have made changes to Armbian to support the NanoPi R2S Plus, including working WAN port and eMMC. These changes have been merged into armbian/build so it is now possible to build with BOARD=nanopi-r2s-plus, to build images for current and edge kernels.
When I started I was not familiar with armbian/build, u-boot or the rk3328 boot process. I didn't find much that gave a good overview, to put all the bits into context. At least, not in terms that I could understand and feel confident I understood them correctly.
The RK3328 Datasheet Revision 1.1 Mar. 2017 should be a reliable source but some of the English is a bit unclear.
The RK3328 has internal memory including BootROM (and Internal SRAM (36KB), utilized for the initial stages of boot, including initialization of DDR RAM and loading the subsequent boot programs.
The BootROM is, as far as I can tell, truly ROM. I found no indication that it can be updated.
The Friendly Elec Wiki page for NanoPi R2S Plus describes the Unbricking Method. This is the worst case means to load software to the system, if it is not bootable by any other means. Systems based on the RK3328 (and several other RockChip SoC models) are often described as 'unbrickable'. What this means is that no software update can make the system completely unbootable. This is because to BootROM is truly ROM: it can't be updated to invalid code. And it supports loading software via USB OTG interface, regardless of the state of any other memory. So, as long as there isn't a hardware fault, the system can be booted and updated, even when it is otherwise unbootable.
Unfortunately, Section 12 Unbricking Method of the Wiki page begins with:
If the ROM is not installed correctly, causing the development board to become bricked, and you might not have the opportunity to reinstall the ROM via an SD card, you need to enter Maskrom mode to unbrick it by erasing the storage device.
This mislead me into thinking that the ROM could be updated to an invalid state. I spent a long time searching for information about how to update the ROM, but never found any.
My conclusion is that the BootROM is truly ROM: it cannot be modified. There are no images to be installed. There is no way to 'reinstall the ROM' via an SD card or otherwise.
The code that can make the system difficult to boot ('bricked') is that on the eMMC. The SD card can always be re-flashed on another system, with a good image. Or the SD card can be replaced. But the boot code in the BootROM checks for bootable code on the eMMC before it tries to boot from the SD card. If it finds data on the eMMC consistent with it containing bootable code, then it copies the code to RAM and executes it. If the code is bad, then the boot will fail and boot from the SD card will not be attempted, even if it contains good code. The Unbricking Method allows the eMMC to be completely erased, in which case the boot code on the BootROM will find it empty and will then attempt boot from the SD card. Thus the system becomes 'unbrlcked'.
The Rockchip RK3328 Technical Reference Manual Part 1 Revision 1.2 July 2017 (I haven't yet found a Part 2), describes the boot process in a bit more detail, in Section 1.2 System Boot.
RK3328 provides system boot from off-chip devices such as SDMMC card, eMMC memory, serial nand or nor flash. When boot code is not ready in these devices, also provide system
code download into them by USB OTG interface. All of the boot code will be stored in internal bootrom. The following is the whole boot procedure for boot code, which will be stored in bootrom in advance.
The boot code in the BootROM 'which will be stored in bootrom in advance' is, as far as I can tell, written once, by Rockchip, as part of SoC manufacturing. It is not possibly to modify it via the USB OTG interface.
The initial boot code checks for an ID Block at a particular location on each of the bootable devices. I don't yet know what the ID Block content is or where it is stored, but Armbian builds SD card images that contain it, so it shouldn't be too difficult to find.
The following devices are checked for an ID Block:
- eMMC
- external SPI Nor Flash
- external SPI Nand Flash
- SDMMC card
The devices are checked in this order and code is loaded to SDRAM from the first one with a valid ID Block, and that code is executed. Subsequent devices are checked only if preceding devices do not have a valid ID Block. This is how bad data on the eMMC can 'brick' the system: if the eMMC has a valid ID Block but does not have good boot code, the boot will fail.
If none of the devices have a valid ID Block then the USB OTG interface is initialized and this can be used to load a program to SDRAM. There are programs available that allow the various devices to be erased and written with new images. It is sufficient to simply erase the eMMC, external SPI Nor Flash and external SPI Nand Flash, in order to be able to boot from the SDMMC card. In fact, the NanoPi R2S Plus does not have external SPI Nor or Nand Flash, so it is sufficient to erase the eMMC, unless the system has been modified.
What the Technical Reference Manual doesn't mention, in its 'whole boot procedure for boot code' is that there is a way to skip checking the external devices for valid ID Block and jump straight to initialization of and loading code from the USB OTG interface.
Rockchip has a Wiki page Boot Option, that describes the boot procedures in somewhat more detail but it barely mentions the option to boot via USB OTG interface. Rather confusingly, in section Boot from eMMC, it describes how to update the eMMC via the USB OTG interface. It provides sample commands but almost no explanation of what the requirements are or how the commands work. But there is a link to 'Get the board into maskrom mode'
This Rockusb wiki page provides some description of this other boot option supported by the code in the BootROM.
The NanoPi R2S Plus provides a convenient 'Mask' key, to boot into Maskrom mode regardless of whether any of the usually checked storage devices have a valid ID Block. If this button is pressed while the system is booting up, the BootROM code will not find a valid ID Block on any device and will proceed to initialize the USB OTG interface and wait for commands there.
It looks like the Reset button holds eMMC Data 0 line low. Presumably this prevents the boot code from reading a valid ID Block from the eMMC. But what about the SPI Flash and SD card? If the button only affects eMMC, then if there is bootable code in those other devices when there is unbootable code in eMMC, then holding the Reset button may result in boot from one of those other devices, rather than USB OTG. Presumably this is why the Unbricking Method specifies to remove the SD card. As NanoPi R2S Plus doesn't have any SPI Flash device, if the SD card is removed and eMMC access is effectively disabled by holding Data 0 line low, then the boot program will not find any valid ID Block on any device and will proceed to the USB OTG interface.
Details of how the next stage of boot program is loaded from eMMC or SD card are in Wiki Boot Option and can be deduced from how Armbian prepares SD card images and how the armbian-install utility writes the eMMC for boot from eMMC. The U-Boot documentation also describes how images can be prepared - there are several options.
Being unable to modify the BootROM code is a limitation of the SoC. Not a problem as long as it works but if there are any bug or limitations in the SoC BootROM code, then the only solution will be to get a new SoC (practically, a new board) with a different, improved BootROM. For example, I see some reports of the BootROM code not handling SPI Flash very well. Not much of an issue for NanoPi R2S Plus, as it doesn't have wiring for SPI Flash, but NanoPi R2S has such wiring, despite it is delivered without a chip installed. But, if an SPI Flash chip is installed, then one must work around the limitations of the BootROM. In particular, I saw reports that if a large SPI Flash is installed, the interface switches from 3bit to 4bit address mode but reset doesn't switch back so on reset the SPI Flash cannot be accessed. Only a cold boot allows boot from SPI Flash.
The BootROM code is proprietary and not made available by Rockchip. But Tom Trebisky and Marcin Wozniac, at least, have extracted the BootROM code for investigation.
See ig3/trebisky-Rockchip and ig3/MarcinWad-RK3328-Uboot-SPI for clones of their GitHub repositories, including RK3328 BootROM images, and some cryptically documented code for extracing the BootROM code in the former.
See also dmkikushin-RK3328-Uboot-SPI for a somewhat enhanced version of Marcin's repo.
Marcin described some of his difficulties getting NanoPi R2S to boot from SPI Flash in his post to the u-boot mailing list 2022-12-22.
From u-boot Thu Dec 22 20:15:53 2022 From: =?UTF-8?Q?Marcin_Wo=c5=baniak?= <marcin.wadowice () gmail ! com> Date: Thu, 22 Dec 2022 20:15:53 +0000 To: u-boot Subject: Rockchip RK3328 4-byte addressing problem in SPI Message-Id: <f3451d5f-7b3b-e393-4ea7-370a86d45458 () gmail ! com> X-MARC-Message: https://marc.info/?l=u-boot&m=167175500517093 Hello, I would like to let you know how i've lost about 2 weeks because of lack of Rockchip BOOTROM documentation. I tried to use Mainline Uboot to Boot NanoPI R2S using soldered SPI Winbond 25Q256 chip. I had to make some patches in Uboot core files to make it work. First one adds SPI boot device. File |arch/arm/mach-rockchip/rk3328/rk3328.c with adding: | |[BROM_BOOTSOURCE_SPINOR] "/spi@ff190000", |const char * const boot_devices || Second was |arch/arm/mach-rockchip/spl-boot-order.c and last if statement:| |if (!uclass_get_device_by_of_offset(UCLASS_SPI, node, &parent)) return BOOT_DEVICE_SPI;| || This patch is added because RockChip SPI driver is using UCLASS_SPI instead of SPI_FLASH so boot device wasn't connecting with right DM Driver rk_spi.c Third one fixed SPL and it was added at the end: |drivers/spi/rk_spi.c | |DM_DRIVER_ALIAS(rockchip_rk3288_spi, rockchip_rk3328_spi);| After that board booted successfully. Second bigger problem was a mistake and lack of RockChip Bootrom documentation. I've soldered 32Mbyte Flash and didn't know that U-Boot in SPL was switching it's 3-byte address mode to 4-byte one to gain access to full 32 Mbytes. The problem was that board booted in cold-start but after issuing "reset" - it was going to MASKROM mode like no SPI was soldered or empty. After loosing two weeks i figured that RockChip bootrom talks to SPI using only 3-byte addressing. So leaving 4-byte switched by UBoot SPI chip made it unusable to RockChip Bootrom. I found this by dumping Bootrom and decompiling it. I don't know if it's enough information to make a patch to MainLine (at least for SPI boot in RK3328). Problem with 3-byte addressing is a complete different story. Sorry for my "unproffesional" mail. I am writing this type of message for a first time. Marcin
Follow the thread for all the followups but note this from Kever:
Sync with engineer working on these area, and get below: Yes, this "4-byte addressing problem in SPI" issue is in SoCs including rk3328, this only happen on SPI NOR size at least 32MB, most of customers only use smaller size SPI NOR. If 32MB+ SPI NOR is needed for the project, them we would suggest to use those SPI flash with 3&4 byte command set support; or else workaround to do the snor_deinit and exit_4byte_address_mode will needed before every reboot/hot reset in the system.
And a further response from Michael:
I'm not sure this is a problem of the SoC (bootrom). The issue is either board design or flash usage pattern. If you enable 4byte mode and don't have a hardware reset pin (or don't assert it during board reset) you are likely in trouble. Have a look at JESD216D. The 16th DWORD lists 6 different software reset methods, there is even a "no software reset instruction is supported". That's also true for the "exit 4byte mode": there is no common way to exit it. You could try to read the SFDP tables. But that will only work for flash devices with (valid) SFDP. So in the general case, you are probably f** when you enable the 4 byte mode, boot from that flash and don't have a reset. As mentioned here, you should instead use the 4byte opcodes where possible. -michael
These details aren't important for my NanoPi R2S Plus, but they illustrate why it is sometimes important to have complete information about the board, including the boot code and how it initializes hardware. For the RK3328 SoC, it seems the only way to get the full details of the boot code is to extract it from the relevant system, disassemble it and figure out what it is doing, as Rockchip does not provide binaries or source code for the BootROM code.
It would also be interesting to know:
- If there are different versions of the BootROM code
- What, if anything, the BootROM code writes to console
- How it initializes the SoC and external devices
- How it checks for the ID Block
- How it loads the next stage software and executes it
- The full implementation of the USB OTG interface