Continuous Integration for Embedded Systems

• Tools
• Software Development
• Project Management

It is no secret that anyone who wants to streamline project management, reduce risk and improve the quality needs some form of "automation" in SW development processes. What is commonly used in most companies as a tool for such automation is called Continuous Integration (CI). It is a good practice for embedded systems as well even though it is much harder to use CI for embedded systems compared to pure software development because embedded systems mostly depend on the hardware they are running on.

Anyhow, there are different types of embedded CI and one should be careful about choosing the right type and necessary plug-ins of available CI tools to strike a balance in quality and speed. Major types of embedded CI are "host", "non-host" and "hardware in loop". 

Regardless, typically a CI server executes the following steps in the most basic scenario without interacting with any HW at all:

                   
These steps are ideally triggered on every code commit to the source code repository (git, perforce, svn, etc). The reasons and benefits of running this process for each code commit  are

a) to get early feedback on whether the build is broken or not. It becomes important when working on teams, especially the ones that are geographically distributed. 

b) to avoid scenarios where the code runs just fine on developer's machine/setup but not in the field since it goes through a clean build with lots of checks. 

c) to provide quick access to QA/System test team for binaries to let their test automation start their smoke,sanity and regression testing without depending on manual binary delivery by developers

A more advanced CI process would have some more steps to control quality such as the following:

In the static code analysis step, tools like Cppcheck, Clang, Flexelint, Pylint, etc are used. Code quality checks are done both bythe static code analysis tools partially and by home-grown SW unit tests and scripts to mandate certain processes in development.

Flow for non-host based CI process would be as the following:

Of course, cost of host-based CI testing is more expensive since every test runner would require a physical HW as well as a JTAG debugger and a serial port adapter to collect reports. It then also limits the number of tests that can run in parallel. 

CI processes can be more advanced to extend the testing to system integration testing with hardware-in-the-loop (HIL). In HIL testing,  instead of running SW unit tests on the target device, the actual production FW is run and synthetic data is sent to the device via an interface ETH, USB, CAN-bus, Serial port etc and processed data output is recorded simulating a display to compare the expected and actual results. Note that in this case, since events are driven from the PC side, timing accuracy would be low. This kind of testing is mostly done for verification of correctness of the functionality of the FW rather than performance or to find timing issues in the platform.   Benefit of such testing is that this kind of testing is more comprehensive than SW unit testing in a sense that final FW with actual target HW is in the loop during testing. 

There are multiple choices of CI tools such as Jenkins, Travis-ci, Bamboo etc in the market ranging from open-source to paid products. Jenkins is my favorite as it provides hundreds of plug-ins to support building, deploying and automating any project including embedded SW projects.

There is a series of articles on Papirux that goes through how to set up Jenkins on Ubuntu and integrate it with GitHub, CHECK SW unit test framework, CppCheck static code analysis tool, MemCheck memory check tool and provide some information regarding how to set up a build-system from scratch and develop sanity checks around it with Jenkins. You can sign-up for free on swe.papirux.com to read the how-to article series.

Sign in