Unit testing frameworks for embedded c

Until now, I was not be able to arrange a full unit testing for any of 
my projects, but I want to invest some time to learn this method of 
development.

There are plenty C unit testing frameworks out there. I think the most 
used are Unity[1] and Cpputest[2]. It seems to me others aren't oriented 
to embedded and/or C language (but C++).

I read many comparisons among unit testing frameworks, but sincerely I 
can't decide which one to use.

Do you use any? Which one and why?




[1] http://www.throwtheswitch.org/unity

[2] https://cpputest.github.io/

Am 26.01.2022 um 13:23 schrieb pozz:
> I read many comparisons among unit testing frameworks, but sincerely I
> can't decide which one to use.
> 
> Do you use any? Which one and why?

None. Just <assert.h>.

For me, the main objective for a unit test framework is to have some
syntactic sugar for assertions and, less important, for grouping tests
into test suites and stuff.

Do we need assertions? It is surely helpful, if an equality assertion
fails, to have expected and actual value presented on a golden plate. On
the other hand, getting those values with a debugger isn't much more
complex, and you typically fire up a debugger anyway to investigate. In
a language such as Java or C++, you can have some higher-level
assertions such as "assert-throws" or "assert-container-contains", which
save quite a lot of typing. In C, the highest level you could get is
something like "assert-string-equals", but the gain compared to plain
'assert(strcmp(...) == 0)' is vanishingly small.

Do we need to group tests? People delight in numbers, "I have 937 unit
tests, and of those, 936 pass, look how awesome I am!". But, seriously,
even if one test fails, the product is broken and someone needs to
investigate, so, for me, unit tests produce a "ok"/"nok" result, so I
just need a binary that runs or crashes.

Now, what I don't expect in a unit test framework proper is the whole
area of mocking stuff. In languages like Java, how to mock something is
pretty straightforward: you use abstract interfaces, factories, etc.,
and to mock a file system access, you pass a mock FileSystem object. In
C, how you can mock - if you mock at all - depends on your codebase. Do
you have the equivalent of virtual interfaces? Do you want to play
linker tricks? Do you want to play preprocessor tricks? That decides
what mocking framework, if any, you can use.

Personally, when I unit-test C, I'm testing computations and logic,
which usually don't need mocking. Things like: put in this input, expect
this output, from my computational kernel.


  Stefan

Il 26/01/2022 18:10, Stefan Reuther ha scritto:
> Am 26.01.2022 um 13:23 schrieb pozz:
>> I read many comparisons among unit testing frameworks, but sincerely I
>> can't decide which one to use.
>>
>> Do you use any? Which one and why?
> 
> None. Just <assert.h>.
> 
> For me, the main objective for a unit test framework is to have some
> syntactic sugar for assertions and, less important, for grouping tests
> into test suites and stuff.
> 
> Do we need assertions? It is surely helpful, if an equality assertion
> fails, to have expected and actual value presented on a golden plate. On
> the other hand, getting those values with a debugger isn't much more
> complex, and you typically fire up a debugger anyway to investigate. In
> a language such as Java or C++, you can have some higher-level
> assertions such as "assert-throws" or "assert-container-contains", which
> save quite a lot of typing. In C, the highest level you could get is
> something like "assert-string-equals", but the gain compared to plain
> 'assert(strcmp(...) == 0)' is vanishingly small.
> 
> Do we need to group tests? People delight in numbers, "I have 937 unit
> tests, and of those, 936 pass, look how awesome I am!". But, seriously,
> even if one test fails, the product is broken and someone needs to
> investigate, so, for me, unit tests produce a "ok"/"nok" result, so I
> just need a binary that runs or crashes.

I mostly agree with you.  Indeed I think it's not important which 
framework you use, but if you use and how you arrange unit testing at first.

The big issue with C embedded projects is how you organize your build 
system to (maybe it can be an entire discussion of a new post):

- build Release/Debug for multiple embedded targets and models of final 
product
- build the same as above, but for host machine (simulator)
- build and run tests on the host
- (eventually) build and run tests on the targets

This is more difficult for embedded because we use different 
cross-compilers for multiple targets, multiple IDEs from silicon vendors 
and so on.

This is my main reason why migrate to a Makefile and abandon automatic 
build system of IDEs from silicon vendors.
Starting from a custom and ubiquitous (even if cryptic and ancient) 
Makefile, I can customize the entire build system for production on real 
targets, simulators, tests and so on.

However this is very difficult for me and starting from a working and 
non trivial example is a big bonus.

I found this article[1] very very interesting in this topic. It uses 
Cpputest in examples (even if the author clearly says that the framework 
is not important).
	
The author gives a pro to frameworks for code coverage report. I know 
almost anything on this, so I don't know if code coverage can be 
reported without a unit test framework.


> Now, what I don't expect in a unit test framework proper is the whole
> area of mocking stuff. In languages like Java, how to mock something is
> pretty straightforward: you use abstract interfaces, factories, etc.,
> and to mock a file system access, you pass a mock FileSystem object. In
> C, how you can mock - if you mock at all - depends on your codebase. Do
> you have the equivalent of virtual interfaces? Do you want to play
> linker tricks? Do you want to play preprocessor tricks? That decides
> what mocking framework, if any, you can use.

Mocking is another argument that I will certainly explore. I will start 
testing simple units that don't have many dependecies and where I can 
use stubs or fakes, not real mocks.


> Personally, when I unit-test C, I'm testing computations and logic,
> which usually don't need mocking. Things like: put in this input, expect
> this output, from my computational kernel.

This will be my starting point too, but I think it is a small part of a 
non trivial project.
Critical and obscure bugs are hidden in complex units that have many 
dependencies that can't be replaced by simple fakes and stubs during 
tests, you need full mocks.






[1] https://interrupt.memfault.com/blog/unit-testing-basics