As part of my effort to create a new crowd-sourced #FAQ section and also with the intention to help promote my good friend Jacob Beningo's upcoming webinar titled Secure Bootloader Design Techniques for MCU's, I am today starting this new thread about Bootloaders.  

The hope is that by the time you guys are done with sharing your thoughts here, this thread will have become a very useful resource for Embedded Engineers to read some insights about #Bootloaders - what they are, why we have them, when they are useful, etc.  

As always, thanks a lot for your time!

This is a major topic, but answering the basic questions will give a good overview.

Bootloaders, 

what are they?

- A bootloader is a small OS, or application, designed to download firmware in the embedded device (MCU's internal or external memory). Gives you the possibility to upgrade or change your systems firmware.

why we have them?

- It is beneficial to be able to update a deployed system (to fix bugs, or just to update with new features) without actually attaching a JTAG adapter to the PCB. Access to the pcb itself is in many cases not possible, and the connector might not be even on it, since the pins are used for something else.

when are they useful?

- As mentioned above, the bootloader is needed when you need to have the possibility to update your firmware without having access to the programming header on the PCB and without the need to re-enable the security settings to allow this.

where are they stored?

They reside in the ROM or in the FLASH memory of the MCU, usually write protected, so you can't destroy them (but don't force your luck too much).

when do I need one?

Whenever you want to have the possibility to upgrade your firmware via usb/serial/tcp-ip/wireless, and not the classical JTAG programming adapter.

How to implement one?

There are many open source bootloaders available, from compact ones intended for 8 bit platforms, to complex linux based systems. There is not a real need to start implementing one from scratch.

Bootloader flavors:

Wired connection -  most known might be the Arduino bootloader, when you download a new binary (compiled sketch) via the USB, the download itself is done via the bootloader.

Wireless connection - They are very common, you might have heard the term update over the air, that feature is also based on a bootloader, just the data transfer is made wireless.

Secure bootloaders - these transfer the data in an encrypted form, so no tampering or spying on the content is possible. The implementation choices are quite variate, usually in the initial phase a mutual authentication takes place, this can be based on symmetric or asymmetric encryption. For extra security, the mutual authentication is made with asymmetric cryptography and symmetric session key is exchanged, which is then used for the bootloading data transfer (symmetric decryption is way faster).

Hope it helps.

Additional thoughts, borne of much experience.  This mostly applies to very small, deeply embedded processors:

• If you have a custom protocol (which you should think hard about unless it's tiny) then you'll need a custom boot-loader to talk it.
• You really don't want bugs in your boot-loader, and every line of code is an opportunity for bugs -- keep it simple!
• If you find someone's example code on the web, use it as an example only.  Because you really don't want bugs in your boot-loader.
• If you're working with a tiny processor, then if your app code hangs you won't be able to get back to the boot-loader -- so make sure there's a "get out of jail" jumper that forces the boot-loader to not run app code if it's shorted on power-up or reset.  This can be as primitive as checking the state of an unused pin on the microprocessor (if the board guys forgot to give you one) or as classy as a push-button on the board.  I prefer a jumper, or an uninstalled 0603 resistor that I can short out with something metallic.
• It's a Very Good Idea to have end-to-end app code verification -- I like tacking a CRC on the end of my application code space at build time (there's tools to do it), that the boot-loader verifies on startup before jumping to app code.  That way, the wrong app code or a faulty or interrupted download won't shoot you in the foot.

I like to keep "bootloader" for the ROM and use "loader" for the code I write.

I have worked on a number of loaders for the automotive and aviation industry using CAN, LIN, RS232, ethernet, on a wide range of embedded DSPs and controllers.

I just completed extending a vendor supplied loader for the SAMv71 to use a serial driver. The Atmel AT16743 app note has a nice discussion of generic loader concepts before it gets into SAMv71 specifics.

This app note makes a distinction between privacy, integrity, and authentication which are all important to keep in mind. Privacy is encrypting the firmware and/or the communications. Integrity means using a CRC or hash to verify the update. Authentication is verifying the identity of the sender and receiver of the firmware. It also covers the decision on whether to jump into the app or loader and several ways to make the decision.

In my experience it's always good idea to have the ability to update the loader itself. The loaders I have worked on were setup to run entirely from ram. The complete loader upload is saved in ram until the entire update has been validated. At that point, the loader flash is erased and written. This reduces the window of time when the device could lose power and become a "brick". With aviation, automotive and marine, products using loaders we have never bricked a unit in the field. The loader doesn't need to be updated very often if the design is kept simple and it is well tested. 

I make provisions keeping not just a loader and an application in flash but for multiple components (parameters, models, production calibration values, and at least one reserved space). Each component uses the same structure with it's own version and fingerprint and each is delivered in it's own binary file. (usually a .hex) The GUI and the loader understand this structure and can updates any or all of these components with no additional overhead. The loader only requires the application to be valid to make the jump. It's the applications responsibility to validate the other components.

In the early days, we used a hardware pin to instruct the loader to stay in the loader. We sometimes put a jumper or switch across a pin in the serial port db9. Lately, we have been relying on a "jump signature" shared memory location between the application and the loader to decide when to stay in the loader. With this approach, there is some risk of locking up the application that you won't be able to set the "jump signature". A locked up application will still reboot on the watchdog and you can have the loader check communication before jumping to the application.

I highly recommend having the capability of reading the version AND the fingerprint(CRC/SHA hash) of every component (bootloader, application, parameters, models, etc...) that reside in flash. Not knowing the exact software resident in flash is a recipe for confusion. My first two questions when debugging a controller are is it plugged in and what versions are on it.

One of the other posts mentions the loader initializing memory, gpio, etc. We have the loader test and initialize what it needs but the application is responsible for initializing everything it uses. It doesn't need to retest memory checked by the loader but it does need to initialize everything it uses. The application can not rely on power up values nor the state the loader may have set up. This is useful when you use a debugger which may not go through the loader and it keeps the loader requirements untangled from the application.

Development Stories

I was involved in one project where the loader worked fine until just prior to a production deadline. The loader was tested early in the project when the application was relatively small. During development the developers needed to use a jumper to disable the watchdog to use the debugger. The size of the application grew from one sector to five. During the final tests, the watchdog jumpers came off and the watchdog would occasionally reset during the application sector erase. At this point many of the controllers had passed the hardware quality tests and were sealed up. Being about to update the loader to refresh the watch dog during the sector erase saved a lot of expense. It also taught me to always fill and test the application at maximum size.

Keep in mind the field conditions and always test the bootloader under worst case conditions. I have worked on shipboard CAN interfaces with extremely long cables and aviation systems mounted near strong motor interference. Being able to slow down the baud rate can make a difference when conditions get bad or the customer has a slow laptop. But slower isn't always better. Bit rate accuracy is critical. I fixed one project that had been using 115.2K serial communications and then switched main crystals in the DSP. The new crystal didn't divide down exactly to 115.2K and boot loader communications weren't 100% reliable. Most the time it would work, but you didn't know it failed until the entire application was downloaded. We suffered for quite a while before finding the cause and changing the baud to 128K which had no frequency error.  The communications were more reliable at 1024K than at 115.2K because of the clock divide accuracy error. Make sure your communication setup is as accurate as possible. Clocks should divide exactly to the baud rate you want to use. 

The ability to update the loader saved me on a Marine application. The controller used the internal oscillator and the original loader did not perform temperature compensation. It "worked" until you added long communication cables and temperature extremes when the loader started dropping bits. 

And if you are dealing with CAN it isn't just picking a 500K bit rate. If another company or group is writing the loader UI you need to know what sample point they expect and choose the appropriate sample method and jump width sync to keep the loader functional over a wide range of termination, cabling, and environment conditions. It's not trivial to pick all the CAN quanta to get the UI PC and embedded device at optimal settings.  But before your loader goes out into the field you need to assure the bit rate accuracy and all parameters give you as much margin as possible for clocks that change with temperature and cable capacitance.  It may look funny to have a long coil of cable that could run the full length of a ship under your desk but using representative cables is important to CAN and communications in general so get your development setup as close as possible to the field install.

I would also stress keeping the loader requirements simple and reducing User interface opportunities for errors. The Atmel GUI with AT16743 loader reads the name of the .bin file and has the user enter the starting address. What were they thinking? The starting address exists in the .hex file. Why would you give the user the opportunity to make a mistake? Why would you add another UI item that needs to be tested?

An advanced bootload subject is what to do with when you have multiple devices on a bus or multiple buses. Multiple devices on the same bus are the norm for CAN. What happens to the other devices during updates, do they keep talking? A further complication is that some devices may be on multiple CAN buses but the UI for update may only have access to a subset. With careful planning, you can update everything on your network via a single bus. You may also wish to simultaneously update multiple devices. This may not seem important until you have dozens of devices on the bus and minutes per update with expensive systems and labor the costs add up.

Last but not least, think about manufacturing. You need to get the loader into the controller at least once. In the old days, we would provide programmed EPROMs. Today most systems have flash internal to a SMD micro. As an engineer you will probably flash the loader via an emulator. But what are they going to use on the factory floor the first time? Most microcontrollers come with a basic bootloader in ROM that can boot from serial, CAN, etc.  Make sure your hardware guys make these pins easily available during manufacturing builds. The ROM bootloader may require manipulating certain pins. Make sure these pins are available during manufacturing. Don't expect a lot of quality or support for the ROM bootloaders. Expect garbage and you won't be disappointed. This is probably the biggest reason not to depend on the ROM bootloader from the silicon vendor. Make sure you test the provided bootloader utilities, because they probably won't be improved. I have been on multiple projects where ancient PCs running XP were kept around for years because they were the only machine that could talk to the ROM bootloader.

Additional thoughts to @Tim Wescott and @Laszlo, based on my experience with 32 Bit ARM processors and Linux :

Bootloaders are considered a necessity if you are working with 32 Bit Processors running an OS like embedded Linux. 

Bootloaders do the following in a Linux based embedded product 

1. Initialize the RAM and Flash. 

2. Initialize the GPIOs - Most of the designs require some GPIOs to be controlled as early as possible on power up. 

3. Initialize any peripherals that are required at startup - this depends on your use case. Normally we initialize the LCD and put up a boot LOGO and indication that the device is ON and is getting ready. Normally this happens within a second. The Linux kernel takes more time to boot. 

4. Copy the Kernel from Flash/Storage to RAM (Normally we try to make the Flash and RAM operate at MAX speed to reduce the boot time) and jump to the kernel start 

Bootloader was traditionally only to load the kernel onto the RAM and start. Now, practically a few other important functions are added based on requirements 

I see bootloaders as inevitable in embedded OS based systems for the following main reasons. 

1. On field firmware updates - When we update a device on the field (your routers do all the time), the OS kernel, device drivers, applications, config files etc. may get updated. Whenever the OS or drivers are updated it is updated from the Bootloader. We even implement fail safe firmware updates by keeping back up copies of Kernel binaries. As a thumb rule, normally we advise Bootloaders never be updated on the field. 

2. Boot indications and early initialization of certain peripherals, boot decisions - The bootloader is very essential to give early indications to the User that the device is ON and loading.. 

3. Battery Monitoring - Also in bootloaders, we read battery capacity to decide whether the boot should continue or indicate to the user that the battery is low to boot and device has to be charged. 

There have been a lot of very good replies here, but here are a couple of other points and opinions as to what maybe necessary in a bootloader.

1) For tiny processors,there may be further constraints on available memory.  I have been using a lot of AMTEL(MicroChip) AVR parts.  In some of my projects the size of the bootloader has dictated the overall size of the part.  In one project in particular, I needed to double the size of the part in order to have enough bootloader space (2K) to support the customers desired feature set.

2) Bootloaders are not limited to digital interfaces.  On a recent project the customer wanted to be able to upgrade the firmware with an analog input, so a cell phone could be used (no PC interface on the device).  FSK modulation was used to send the ones and zeros to the bootloader.  The bootloader decoded the analog stream and updated the FLASH, all is 2K of memory.

3) Over-the-air downloads are pretty cool, but they can also be pretty slow in a ZigBee mesh network, where it is important to leave plenty of network bandwidth to support normal communications while updating a few devices.  On one system that I worked on, it could take hours to update the entire network on a floor, with several floors in the building.  A technician would have to kickoff the update process and then wait hours (or days) for all modules to update.  Since time is money, an alternate process was developed, where in images (current and upgrade) were compared and only the differences were transmitted.  Each module would construct a new image in FLASH, based on the current images (also in FLASH) and the changes.  When complete, the new image would be copied into the target FLASH location and the devices could be rebooted.  This process could reduce the upgrade process into minutes.

Until it is possible to generate bug free software, or software developers are given the time required to get the software complete (all the required features in the first shipping release), bootloaders will be in our future.

Enjoy!

Gene

Gene!  So good to see you back.  Thanks for taking the time to share your thoughts.

For those of you who don't know Gene, he has several great blog posts here.

These are documents of the boot-loaders in the uTasker project (focusing on Kinetis Cortex-m0+ and Cortex-m4) but usable on several families (Stellars, Atmel SAM, AVR32, STM32, Coldfire, LPC2xxx)
http://www.utasker.com/docs/uTasker/uTasker_BM_Loa...
http://www.utasker.com/docs/uTasker/uTaskerSerialL...

The "Bare-minimum" loader is typically 2..3k in size and allows updating application, and/or stand-alone loaders in a fail-safe manner based on intermediate storage (internal Flash or SPI based Flash) and optional encryption.
The "Serial loader" is a versatile stand-alone loader using UART (Kboot, SREC, iHex, Modbus), USB (MSD, CDC, memory-stick), SD-card or Ethernet (web server loader, Modbus) occupying between 6k and 18k and allows mixing of different types and optional encryption.

"BM" and serial loader can be used together to allow field-updating of application and serial loader.

The documents give details and flow diagrams of all operations.

At the end of the serial loader document there is also a discussion of things to be aware of to ensure that application are compatible (eg. the requirement for correctly locating their interrupt vectors). Generally, if the application respects a few basic rules, the boot loaders can be used with any application.

Some of the newer Kinetis chips include a ROM based boot loader but this tends to be much less flexible than adding one's own.

A stand-alone boot loader should always run when there is no application installed. Otherwise strategies for moving to the boot loader include (rather than immediately starting the application) are based on inputs (eg. a switch), time (eg. a short delay to received a command to stay in the loader mode) or a mail-box between application and loader so than an application can command that the loader be executed.

USB-MSD loading tends to be the most popular method at the moment since it is fast and practical and doesn't need a special program on the PC; USB-MSD is compatible on Windows, MAC and Linux.

All code and simulations of the operation are available as open source (for Kinetis) on GIT hub.

Regards

Mark

Bootloader is a basic assembly code which starts when your board is powered up.Mostly it boots from a non volatile memory like flash or ROM.When a board is powered up the program counter is at a reset vector of the specific architecture and the bootloader code is linked to this reset vector (This is done through a linker script).This is how a bootloader gets the control at the startup. After getting the control it will initialize all the processor  peripherals, registers and set up the C stack. Mostly there are two types of bootloaders:-

1)Primary bootloader(PBL)also called secure bootloader:-

This bootloader is inside the non-volatile memory(ROM) and its main purpose is to initialize the architecture and the microprocessor/micro-controller peripherals.Also in case of a secure bootloader it verifies authenticity of the chip using an encryption algorithm like RSA or Key ladder.

2)Secondary bootloader:-

The secondary bootloader resides in flash and it is the job of primary bootloader to load the secondary bootloader in RAM.The main job of secondary bootloader is to setup the C stack and initialize everything needed to boot the Operating system ,load the Operating system and give the control to the Operating system.

One important consideration for a boot loader is if it needs to update itself, not just the application.  If this is required, there are a number of ways to do this.

1. If there's enough RAM available, the boot loader can copy itself to RAM before updating the flash image.  Many compilers support copying code to RAM for execution.
2. If there's not enough RAM but enough flash is available, two copies of the boot loader can reside in flash.  The regular boot loader will update the second flash copy and then jump to the second copy so that it can update the first flash copy.
3. An alternative to #2 is having a boot loader function in the application that will only update the boot loader in flash.  A menu or button selection in the user interface will cause a jump to the boot loader function which will then receive the new boot loader code over USB and write it to flash.  This is how my company does it so it's a proven method.

The bootloader is the first program to run after power up or reset. Its primary goal is to ready the system for control by the operating system. This requires hardware initialization and choosing the correct image to load from flash. I would like to stress that in today’s connected world, it also needs to allow the firmware image on the system to be updated easily as well as *securely* over wired or wireless interfaces without any special hardware such as JTAG programmer. And this update process must be secured against unauthorized changes in images, flash corruption, communication errors, power failures etc. 

In other words, the bootloader becomes especially critical in the context of IoT devices, the market for which is increasing rapidly. A proper implementation of secure bootloader can eliminate an entire class of vulnerabilities by ensuring that only authorized and integrity-checked code with the correct digital signature is loaded and run. The requirements of a basic secure boot loader are well understood across the industry but the surprising fact is that it is still typically available as a custom implementation for a particular hardware by the manufacturer and differ widely in features included. Reinventing the wheel seems to be the norm when it comes to boot loaders!

How about open sourcing a well-architected implementation of a basic secure boot loader that is easily portable to different OS’s and hardware? It makes sense in terms of cost, security, consistency of management, and flexibility of choice for the customer. The memory footprint and performance of the boot loader is critical in MCU-class devices, given the constraints in embedded systems such as low memory, time-critical computing environment, low-power, fault tolerance, low-cost etc. If the open source route sounds appealing, check out MCUboot (https://mcuboot.com).

thanks,

Aditi

Boot loader is an important part for any embedded product which has at least one communication channel is available.

Through this you will be able to:

1. Diagnose the product.

2. Debug behavior of the product.

3. Helps to correct and upgrade the firmware.

Strictly speaking boot loader does the job #3 mentioned above but in any product it is necessary to have #1 & #2 also integrated with boot loader

Mostly it will be in flash or ROM of the MCU system and it will take control during product power up and it checks for the integrity of the application code and runs the code.

If there are any problems it indicates through some display available in the product and waits for the response in the communication channel.

The same channel can be further used, called as "debug mode", through which one can go little more deep into the problem. Through this one can find the solution for it.

One can attach to the communication channel of the product and find out the symptoms given, analyze and comes with a solution, the next "corrected" firmware can be upgraded through the boot loader.

Many high end processors like ARM cortex M4 etc.. comes with such boot loaders preloaded in ROM and interact through Serial or USB channels which can be thoroughly used in the system development. For example ARM Cortex M4 processor SAM4E16C has boot loader ROM which work in X modem protocol through which you can access the processor and use for debugging an firmware upgrade.

What is a Bootloader?

A program that loads an operating system when a computer is turned on.

That's it.

Now, a modern bootloader for a modern MCU, may incorporate a lot more functionality today, than in yesteryear, and that is where the response from @Laszlo comes in.

I've been in computers forever, well, since the seventies. I worked with an 8008 back then.

Other computers I have worked with, PDP-8, PDP-11, VAX, Alpha, MIPS, 8080, 8085, 8086, 8088, Intel x86 family, Z80, 64180, 6502, 6800, 6809, 68000, 68020. 68030. 68040, ARM, Z8 Plus and many of the Operating Systems that run on those, both commercial and proprietary.