Continuous Integration for Embedded Systems

Dr. Tayyar GUZELSeptember 5, 20172 comments

It is no secret that anyone who wants to streamline project management, reduce risk and improve the quality needs some form of "automation" in SW development processes. What is commonly used in most companies as a tool for such automation is called Continuous Integration (CI). It is a good practice for embedded systems as well even though it is much harder to use CI for embedded systems compared to pure software development because embedded systems mostly depend on the hardware they are running on.

Anyhow, there are different types of embedded CI and one should be careful about choosing the right type and necessary plug-ins of available CI tools to strike a balance in quality and speed. Major types of embedded CI are "host", "non-host" and "hardware in loop". 

Regardless, typically a CI server executes the following steps in the most basic scenario without interacting with any HW at all:

                   
These steps are ideally triggered on every code commit to the source code repository (git, perforce, svn, etc). The reasons and benefits of running this process for each code commit  are

a) to get early feedback on whether the build is broken or not. It becomes important when working on teams, especially the ones that are geographically distributed. 

b) to avoid scenarios where the code runs just fine on developer's machine/setup but not in the field since it goes through a clean build with lots of checks. 

c) to provide quick access to QA/System test team for binaries to let their test automation start their smoke,sanity and regression testing without depending on manual binary delivery by developers

A more advanced CI process would have some more steps to control quality such as the following:



In the static code analysis step, tools like Cppcheck, Clang, Flexelint, Pylint, etc are used. Code quality checks are done both bythe static code analysis tools partially and by home-grown SW unit tests and scripts to mandate certain processes in development.

Here, in the application of CI to embedded systems, there's a distinction between non-host based CI testing and host-based CI testing. In host-based CI testing, SW unit tests are run on target hardware and in non-host based CI testing, SW unit tests or integration tests are on a different platform than the target by compiling the code for x86 instead of ARM, ARC, MSP, DSP etc.

Non-host based CI testing allows to test the business logic and capture Code Coverage reports in addition to reports captured by host-based CI testing. Anyhow, there may be portability issues with non-host based CI testing. On the other hand, in host-based CI testing, CI server cross-compiles SW unit tests, which have been part of a lightweight Unit Test framework to run on target HW, runs the SW unit tests on target HW and collects the reports via serial port or other interface. Therefore, both business logic is tested and HW platform issues are identified.  

Flow for non-host based CI process would be as the following:

and host-based CI process would be as the following:

 Of course, cost of host-based CI testing is more expensive since every test runner would require a physical HW as well as a JTAG debugger and a serial port adapter to collect reports. It then also limits the number of tests that can run in parallel. 

CI processes can be more advanced to extend the testing to system integration testing with hardware-in-the-loop (HIL). In HIL testing,  instead of running SW unit tests on the target device, the actual production FW is run and synthetic data is sent to the device via an interface ETH, USB, CAN-bus, Serial port etc and processed data output is recorded simulating a display to compare the expected and actual results. Note that in this case, since events are driven from the PC side, timing accuracy would be low. This kind of testing is mostly done for verification of correctness of the functionality of the FW rather than performance or to find timing issues in the platform.   Benefit of such testing is that this kind of testing is more comprehensive than SW unit testing in a sense that final FW with actual target HW is in the loop during testing. 

There are multiple choices of CI tools such as Jenkins, Travis-ci, Bamboo etc in the market ranging from open-source to paid products. Jenkins is my favorite as it provides hundreds of plug-ins to support building, deploying and automating any project including embedded SW projects.

There is a series of articles on Papirux that goes through how to set up Jenkins on Ubuntu and integrate it with GitHub, CHECK SW unit test framework, CppCheck static code analysis tool, MemCheck memory check tool and provide some information regarding how to set up a build-system from scratch and develop sanity checks around it with Jenkins. You can sign-up for free on swe.papirux.com to read the how-to article series.


Previous post by Dr. Tayyar GUZEL:
   How to Give Persistent Names To USB-Serial Devices on Ubuntu 14.04


Comments:

[ - ]
Comment by MatthewEshlemanSeptember 9, 2017
to get early feedback on whether the build is broken or not

Absolutely. I'm a big fan and helped setup and push the use of a CI system during a previous life. My thinking is that as soon as a team hits 5+ members working on the same code base, then a CI system quickly becomes invaluable, especially for ongoing confirming the build. Our CI system would automatically email any engineers found to have pushed code that breaks the build. Quick feedback helps alot! 

Thank you for the article!

Matthew

https://covemountainsoftware.com/consulting/


[ - ]
Comment by stephanebSeptember 20, 2017

Thanks for writing this article on EmbeddedRelated.

It looks like a couple of images are missing?

To post reply to a comment, click on the 'reply' button attached to each comment. To post a new comment (not a reply to a comment) check out the 'Write a Comment' tab at the top of the comments.

Registering will allow you to participate to the forums on ALL the related sites and give you access to all pdf downloads.

Sign up
or Sign in