Forums

Remove unused data from the build output

Started by pozz August 8, 2019
I have defined a myassert(char *module, int line) function that uses the 
arguments only in Debug build. myassert() is defined in myassert.c and 
declared in myassert.h.

In myassert.h there's also a macro that tests an expression:

#define MYASSERT(test)		(test) ? (void) 0 :
               myassert(thisModule, (int)__LINE__))

Now in module.c:

#include "myassert.h"
static char thisModule[] = "module";
MYASSERT(always_true == true);


Because myassert() doesn't use its arguments in Release build (it only 
waits in a forever loop waiting for watchdog), I'd like to avoid 
including all thisModule strings in the final output file.

However I didn't find a good solution, except redefining MYASSERT() in 
two different ways, depending on the build, and implementing two 
different versions for myassert for the two builds.

I'm using -fdata-sections, -ffunction-sections and -Wl,--gc-sections. I 
think the linker doesn't remove those strings because they are really 
used in modules and really passed to an external function.
Den 2019-08-08 kl. 15:44, skrev pozz:
> I have defined a myassert(char *module, int line) function that uses the > arguments only in Debug build. myassert() is defined in myassert.c and > declared in myassert.h. > > In myassert.h there's also a macro that tests an expression: > > #define MYASSERT(test)������� (test) ? (void) 0 : > ������������� myassert(thisModule, (int)__LINE__)) > > Now in module.c: > > #include "myassert.h" > static char thisModule[] = "module"; > MYASSERT(always_true == true); > > > Because myassert() doesn't use its arguments in Release build (it only > waits in a forever loop waiting for watchdog), I'd like to avoid > including all thisModule strings in the final output file. > > However I didn't find a good solution, except redefining MYASSERT() in > two different ways, depending on the build, and implementing two > different versions for myassert for the two builds. > > I'm using -fdata-sections, -ffunction-sections and -Wl,--gc-sections. I > think the linker doesn't remove those strings because they are really > used in modules and really passed to an external function.
The typical way to define an assert is to have two variants which depends on the DEBUG variables. The RELEASE variant is empty #if DEBUG #define MYASSERT(test) (test) ? (void) 0 : myassert(thisModule, (int)__LINE__)) #else #define MYASSERT(x) #endif In your module you do: #include "myassert.h" #if defined(DEBUG) static char thisModule[] = "module"; #endif You could locate the static char declaration into a particular segment, and then skip that segment in the release linker command file. The DEBUG qualification could then be replaced by the segment pragma, but it is less obvious. AP
On 08/08/2019 15:44, pozz wrote:
> I have defined a myassert(char *module, int line) function that uses the > arguments only in Debug build. myassert() is defined in myassert.c and > declared in myassert.h. > > In myassert.h there's also a macro that tests an expression: > > #define MYASSERT(test)������� (test) ? (void) 0 : > ������������� myassert(thisModule, (int)__LINE__)) > > Now in module.c: > > #include "myassert.h" > static char thisModule[] = "module"; > MYASSERT(always_true == true); > > > Because myassert() doesn't use its arguments in Release build (it only > waits in a forever loop waiting for watchdog), I'd like to avoid > including all thisModule strings in the final output file. > > However I didn't find a good solution, except redefining MYASSERT() in > two different ways, depending on the build, and implementing two > different versions for myassert for the two builds. > > I'm using -fdata-sections, -ffunction-sections and -Wl,--gc-sections. I > think the linker doesn't remove those strings because they are really > used in modules and really passed to an external function.
There are several things here - some small, some big. To start with, use static const char* thisModule = "module"; With your declarations, the compiler has to allocate an array in ram and copy the string into it at startup (if it hasn't eliminated it due to optimisation - see later). Also consider just using __FILE__ instead of having to declare a module name manually. I don't recommend -fdata-sections unless you are in the habit of including data declarations that are never used by your program (in which case, maybe you want to change that habit). It can reduce the effect of -fsection-anchors, which is an extremely useful option on ARM processors and other RISC devices (except the AVR). I am against the idea of a run-time assertion that leads to a stall and a watchdog reset. It is rare that this helps in real deployments, as there is a high chance of the same state recurring repeatedly, and it gives very little feedback to the developer about the cause of the problem. Logging the fault to NV memory and /then/ reseting is a bit better. And of course, never use a run-time assertion if a static assertion will do the job. The traditional way to handle assertions is something like: #define DEBUG #include "myassert.h" and then have "myassert.h" have: #if defined(DEBUG) #define MYASSERT(test) (test) ? (void) 0 : \ myassert_debug(thisModule, (int)__LINE__)) #else #define MYASSERT(test) (test) ? (void) 0 : \ myassert_release() #endif This would be sufficient to get the effect you are wanting. Personally, I really don't like having the effect of an include file varying depending on macros. A little better IMHO is: // myassert.h #define MYASSERT(test) do { \ if (test) { \ if (DEBUG) { \ myassert_debug(thisModule, (int) __LINE__)); \ } else { \ myassert_release(); \ } \ } \ while (0) This would require the user to define DEBUG before using the macro, but let them change it at different points in the module. It would even be possible to make DEBUG a run-time variable.
Il 11/08/2019 18:57, David Brown ha scritto:
> On 08/08/2019 15:44, pozz wrote: >> I have defined a myassert(char *module, int line) function that uses >> the arguments only in Debug build. myassert() is defined in myassert.c >> and declared in myassert.h. >> >> In myassert.h there's also a macro that tests an expression: >> >> #define MYASSERT(test)������� (test) ? (void) 0 : >> �������������� myassert(thisModule, (int)__LINE__)) >> >> Now in module.c: >> >> #include "myassert.h" >> static char thisModule[] = "module"; >> MYASSERT(always_true == true); >> >> >> Because myassert() doesn't use its arguments in Release build (it only >> waits in a forever loop waiting for watchdog), I'd like to avoid >> including all thisModule strings in the final output file. >> >> However I didn't find a good solution, except redefining MYASSERT() in >> two different ways, depending on the build, and implementing two >> different versions for myassert for the two builds. >> >> I'm using -fdata-sections, -ffunction-sections and -Wl,--gc-sections. >> I think the linker doesn't remove those strings because they are >> really used in modules and really passed to an external function. > > > There are several things here - some small, some big. > > To start with, use > > ����static const char* thisModule = "module"; > > With your declarations, the compiler has to allocate an array in ram and > copy the string into it at startup (if it hasn't eliminated it due to > optimisation - see later).
Oh yes, it was my error.
> Also consider just using __FILE__ instead of having to declare a module > name manually.
Yes, this is a manual declaration, however IMHO it has good benefits. Mostly __FILE__ is expanded to the full path of the source file where it is used and if you use many assertions, it is possible that you can't test the executable on your real target board because of memory limits. Moreover, by using __FILE__ in every assertions, many declarations of different strings will be put in the output file. By manually declaring one module-based string, I'm sure I will have a single string per module. This could be the right time to make another question: why doesn't the linker compare all the strings (or array) and leave in the output file only the strings that are unique?
> I don't recommend -fdata-sections unless you are in the habit of > including data declarations that are never used by your program (in > which case, maybe you want to change that habit).� It can reduce the > effect of -fsection-anchors, which is an extremely useful option on ARM > processors and other RISC devices (except the AVR).
Ok, I got it.
> I am against the idea of a run-time assertion that leads to a stall and > a watchdog reset.� It is rare that this helps in real deployments, as > there is a high chance of the same state recurring repeatedly, and it > gives very little feedback to the developer about the cause of the > problem.� Logging the fault to NV memory and /then/ reseting is a bit > better.
In this case you should arrange an "hidden" assertions log. However if the assertion is fired in the field, you should instruct the user how to read this "hidden" log or you should ask the user to ship the board/product back in factory to read the log. However many times I saw assertions that trigger on some minor bug that doesn't happen repeatedly. With a simple reset, the user see a strange behaviour (reset), but he could continue using it.
> And of course, never use a run-time assertion if a static assertion will > do the job.
Yes.
> The traditional way to handle assertions is something like: > > #define DEBUG > #include "myassert.h" > > and then have "myassert.h" have: > > #if defined(DEBUG) > #define MYASSERT(test)������� (test) ? (void) 0 : \ > ��������������� myassert_debug(thisModule, (int)__LINE__)) > #else > #define MYASSERT(test) (test) ? (void) 0 : \ > ������� myassert_release() > #endif > > > This would be sufficient to get the effect you are wanting. > > Personally, I really don't like having the effect of an include file > varying depending on macros.� A little better IMHO is: > > > // myassert.h > #define MYASSERT(test) do { \ > ����if (test) { \ > ������� if (DEBUG) { \ > ����������� myassert_debug(thisModule, (int) __LINE__)); \ > ������� } else { \ > ����������� myassert_release(); \ > ������� } \ > ����} \ > ����while (0) > > This would require the user to define DEBUG before using the macro, but > let them change it at different points in the module.� It would even be > possible to make DEBUG a run-time variable. > > > >
Ok, thanks for suggestion.
On 12/08/2019 08:55, pozz wrote:
> Il 11/08/2019 18:57, David Brown ha scritto: >> On 08/08/2019 15:44, pozz wrote: >>> I have defined a myassert(char *module, int line) function that uses >>> the arguments only in Debug build. myassert() is defined in >>> myassert.c and declared in myassert.h. >>> >>> In myassert.h there's also a macro that tests an expression: >>> >>> #define MYASSERT(test)������� (test) ? (void) 0 : >>> �������������� myassert(thisModule, (int)__LINE__)) >>> >>> Now in module.c: >>> >>> #include "myassert.h" >>> static char thisModule[] = "module"; >>> MYASSERT(always_true == true); >>> >>> >>> Because myassert() doesn't use its arguments in Release build (it >>> only waits in a forever loop waiting for watchdog), I'd like to avoid >>> including all thisModule strings in the final output file. >>> >>> However I didn't find a good solution, except redefining MYASSERT() >>> in two different ways, depending on the build, and implementing two >>> different versions for myassert for the two builds. >>> >>> I'm using -fdata-sections, -ffunction-sections and -Wl,--gc-sections. >>> I think the linker doesn't remove those strings because they are >>> really used in modules and really passed to an external function. >> >> >> There are several things here - some small, some big. >> >> To start with, use >> >> �����static const char* thisModule = "module"; >> >> With your declarations, the compiler has to allocate an array in ram >> and copy the string into it at startup (if it hasn't eliminated it due >> to optimisation - see later). > > Oh yes, it was my error. > > >> Also consider just using __FILE__ instead of having to declare a >> module name manually. > > Yes, this is a manual declaration, however IMHO it has good benefits. > > Mostly __FILE__ is expanded to the full path of the source file where it > is used and if you use many assertions, it is possible that you can't > test the executable on your real target board because of memory limits. > > Moreover, by using __FILE__ in every assertions, many declarations of > different strings will be put in the output file. By manually declaring > one module-based string, I'm sure I will have a single string per module. > > This could be the right time to make another question: why doesn't the > linker compare all the strings (or array) and leave in the output file > only the strings that are unique?
It should, if "-fmerge-constants" is in use (-O and above). Perhaps -fdata-sections is causing it trouble? (Have you confirmed that the strings are not merged, perhaps by using "strings" on the binary?)
> > >> I don't recommend -fdata-sections unless you are in the habit of >> including data declarations that are never used by your program (in >> which case, maybe you want to change that habit).� It can reduce the >> effect of -fsection-anchors, which is an extremely useful option on >> ARM processors and other RISC devices (except the AVR). > > Ok, I got it. > > >> I am against the idea of a run-time assertion that leads to a stall >> and a watchdog reset.� It is rare that this helps in real deployments, >> as there is a high chance of the same state recurring repeatedly, and >> it gives very little feedback to the developer about the cause of the >> problem.� Logging the fault to NV memory and /then/ reseting is a bit >> better. > > In this case you should arrange an "hidden" assertions log. However if > the assertion is fired in the field, you should instruct the user how to > read this "hidden" log or you should ask the user to ship the > board/product back in factory to read the log.
Of course you need something like that - but the details are well beyond the scope of this thread!
> > However many times I saw assertions that trigger on some minor bug that > doesn't happen repeatedly. With a simple reset, the user see a strange > behaviour (reset), but he could continue using it. >
That is one possibility - it all depends on the balance you need between reliability and the development costs for identifying and fixing the problem. Jet engine controllers and toasters have different requirements here.
> >> And of course, never use a run-time assertion if a static assertion >> will do the job. > > Yes. > > >> The traditional way to handle assertions is something like: >> >> #define DEBUG >> #include "myassert.h" >> >> and then have "myassert.h" have: >> >> #if defined(DEBUG) >> #define MYASSERT(test)������� (test) ? (void) 0 : \ >> ���������������� myassert_debug(thisModule, (int)__LINE__)) >> #else >> #define MYASSERT(test) (test) ? (void) 0 : \ >> �������� myassert_release() >> #endif >> >> >> This would be sufficient to get the effect you are wanting. >> >> Personally, I really don't like having the effect of an include file >> varying depending on macros.� A little better IMHO is: >> >> >> // myassert.h >> #define MYASSERT(test) do { \ >> �����if (test) { \ >> �������� if (DEBUG) { \ >> ������������ myassert_debug(thisModule, (int) __LINE__)); \ >> �������� } else { \ >> ������������ myassert_release(); \ >> �������� } \ >> �����} \ >> �����while (0) >> >> This would require the user to define DEBUG before using the macro, >> but let them change it at different points in the module.� It would >> even be possible to make DEBUG a run-time variable. >> >> >> >> > > Ok, thanks for suggestion.
When making such macros in my own code, I usually have a check first with __builtin_constant_p to see if the test result is known to the compiler. If it is known false, there is no point in calling any assert function. If it is known true, it gives a compile time error as there is no point in waiting for run-time testing.