Efficient debug log

On 27/07/17 10:23, Stef wrote:
> On 2017-07-26 pozz wrote in comp.arch.embedded:
>>
>> I have another situation. I want to include a member in a structure, but 
>> it's only used in debug. This member is used only in debug_printf calls, 
>> so it can be omitted in the Release build. If I use #ifdef DEBUG_ENABLE 
>> #endif to avoid including the member, I will have an error during build 
>> (because debug_print refers to a member that doesn't exist).
>> If I leave the member in the structure in Release build, I will waste 
>> some space.
>>
>> Is there a trick to solve this problem?
> 
> I often find this "waste of space" argument a bit irrelevant. On large
> targets, the few extra bytes usually don't matter at all. On small targets,
> the space argument may seem valid. But often, you want to be able to run
> the debug version on the same target as the release version. This means
> the target must have space for this data and that this space will simply
> be vacant if you have a release version with the data removed.
> 
> There is even a danger of expanding the code in release mode and then
> finding it does not fit anymore when you need to switch back to debug.
> 
> But there may be situations where you have a special (larger) target for
> debug than is used for the release. Or situations where you have to store
> or transmit binaries and space is tight or expensive.
> 
> So for these situations (or others, or just for the fun of it), I do find
> Davids exercise in the other reply very interesting.
> 

When working with very small systems, it is not uncommon to use a larger
or faster device for prototyping, testing, etc., and then a cost-optimal
device for mass production.  Sometimes that is just devices in the same
families with different memory sizes, but it could easily be that your
development and software testing is done mainly on an evaluation board
with a large chip, extra UARTs for debug output, etc., and the final
version will run on a different device.

Even when you have just a single device for everything, you may have
different balances.  Perhaps the release system will hold a log of the
last 24 hours of sensor values, while for debugging it is fine for the
log to be 6 hours and use the extra space for debugging information.

There are lots of possibilities here - even though you are right that
the space cost of an extra debug data item or two is rarely significant.

On 27/07/17 10:06, Stef wrote:
> On 2017-07-24 David Brown wrote in comp.arch.embedded:
>> On 20/07/17 23:14, Stef wrote:
>>> On 2017-07-20 pozz wrote in comp.arch.embedded:
>>>> I usually prints some messages on a free UART used as a debug port. 
>>>> They are very useful during debugging to understand the flow of execution.
>>>>
>>>> Most of the time, I don't compile the debug log facility in release build.
>>>>
>>>> I sometimes need to disable messages from some modules and enable 
>>>> messages from other modules under debugging.
>>>> Moreover I sometimes need to enable/disable less important messages.
>>>>
>>>> Do you suggest some efficient implementation of debug log macros that 
>>>> include all the features above?
>>>
>>> If I have a output stream available for this purpose, I often use the
>>> following (assuming you use C):
>>>
>>> #ifdef DEBUG_ENABLE
>>> #define DEBUG_PRINTF(level, ...) ((level) <= (DebugLevel) ? fprintf(stderr, __VA_ARGS__) : (level))
>>> #else
>>> #define DEBUG_PRINTF(level, ...)
>>>
>>> Call like a normal printf, except there is an initial parameter for the 
>>> level of that paticular call.
>>>
>>> Lower the value of 'DebugLevel' to reduce the output.
>>>
>>
>> Yes, something like that is the way to do it.  That means in the code
>> itself, you avoid having any extra "#ifdef DEBUG_ENABLE" conditional
>> compilation that /really/ messes up the readability of the code.  You
>> just use DEBUG_PRINTF, safe in the knowledge that it does nothing when
>> debugging is disabled.
>>
>> The only unpleasant thing about this method is that if you have local
>> data that is only used for debugging, you are going to get warnings
>> about it when you disable debugging.  Since I like to have most of my
>> compiler warnings enabled, and like my code to be warning-free when
>> compiled, this bugs me.
> 
> This is true, but you could also argue that data only used for debugging
> should be removed as well when debugging is disabled. That will require
> inserting some additional #ifdefs in the code in places where debug-only
> data is used.

You certainly /can/ argue that.  It is not a good idea, usually, to have
code or data that is not in use.  It is a balance of choices, and you
have to pick the method that is clearest, most maintainable, and easiest
to be sure is correct.

> But mostly, i use the construct to print values of variables that are in
> use in normal code. And most often de DEBUG_PRINTF is not disabled at all,
> only the level lowered to a point where only things like startup and
> critical messages are emitted.
> 
>> So I have this definition:
>>
>> static inline int emptyFunctionX(int x, ...) { (void) x; return 0; }
>> #define emptyFunction(...) emptyFunctionX(0, ## __VA_ARGS__)
>>
>> Then when disabling debugging, I have:
>>
>> #define debugPrintf emptyFunction
>>
>> (I don't like defining a function as an expression in the way you did,
>> and I hate all-caps macro names for constants or function-like macros.)
> 
> But this requires you to write an actual debugPrintf function to handle
> the level comparison before calling the actual printf? But I look into
> it, might be a better solution is some situations.

I often have something like "debugPrintf" which would use snprintf to
put the formatted data into a buffer, then send it out on a UART.  It is
typically a good deal more efficient than using a library printf with a
fake file/stream output.

> And i love all-caps macro names for constants or function-like macros ;-)
> 

Many people use them.  Actually, I like that /other/ people use them,
such as in microcontroller peripheral definition headers - it means the
names don't conflict with my identifiers!

And I use all-caps for any macros that are passed via compiler command
lines (I don't use these much, but they can be a fine way to do parallel
builds of different variations of the same code).

But I have never been able to comprehend why someone would think you
should use all caps for:

#define SQRT2 1.4142135623730951

but not for

static const double sqrt2 = 1.4142135623730951;


Or why you should write:

#define VALID_PERCENT(x) (((x) >= 0) && ((x) <= 100))

and

static inline bool valid_percent(int x) {
	return ((x >= 0) && (x <= 100));
}


If the code that calls a "function" cares whether it is a real function
or a function-like macro, then that code is bad or the macro is bad.
Put braces on your if statements, put "do {...} while (0)" wrappers in
your multi-statement macros, never use expressions with side-effects in
your function arguments.


>>
>> "emptyFunction" uses a gcc extension here, but that's a common compiler
>> for many people.  It swallows any arguments without generating code, and
>> without requiring the arguments to be calculated at all (assuming you
>> have optimisation enabled), but considers the arguments "used" as far as
>> warnings are concerned.
> 
> Indeed a common compiler for me as well, but I have a few projects that
> use other compilers.
> 

Yes, me too.  I use gcc of preference whenever there is a choice - it is
one of the criteria I have for picking a microcontroller for a project.
 It is, in many ways, the best choice of tool for most of my work (I am
fully aware that one of the reasons gcc is a good choice for me is that
I use gcc so much).

If I have to use a different compiler, I will often also use gcc (for
any target) in parallel to do extra error checking.  Things like
__attributes__ are wrapped in conditional checks on __GNUC__.

On 07/20/17 09:31, pozz wrote:
> I usually prints some messages on a free UART used as a debug port. They
> are very useful during debugging to understand the flow of execution.
>
> Most of the time, I don't compile the debug log facility in release build.
>
> I sometimes need to disable messages from some modules and enable
> messages from other modules under debugging.
> Moreover I sometimes need to enable/disable less important messages.
>
> Do you suggest some efficient implementation of debug log macros that
> include all the features above?

For any system designed here, some sort of debug and
syslog is included at the design stage. To start, always
an led or two hanging off spare port lines, then a simple
class / value pair syslog facility, with an output to
leds or a uart.

A simple syslog can be implemented in less than a page of
code, with an array of structures for the subsystem class
and value pairs. Everything in the system that can have
an error condition or other useful info has an entry.

Trivial stuff really, but gets the job done...

Chris

Il 29/07/2017 14:47, Chris ha scritto:
> On 07/20/17 09:31, pozz wrote:
>> I usually prints some messages on a free UART used as a debug port. They
>> are very useful during debugging to understand the flow of execution.
>>
>> Most of the time, I don't compile the debug log facility in release 
>> build.
>>
>> I sometimes need to disable messages from some modules and enable
>> messages from other modules under debugging.
>> Moreover I sometimes need to enable/disable less important messages.
>>
>> Do you suggest some efficient implementation of debug log macros that
>> include all the features above?
> 
> For any system designed here, some sort of debug and
> syslog is included at the design stage. To start, always
> an led or two hanging off spare port lines, then a simple
> class / value pair syslog facility, with an output to
> leds or a uart.
> 
> A simple syslog can be implemented in less than a page of
> code, with an array of structures for the subsystem class
> and value pairs. Everything in the system that can have
> an error condition or other useful info has an entry.
> 
> Trivial stuff really, but gets the job done...

Do you have a facility to disable (not present in the final binary) 
syslog messages from single modules?

Do you disable (as before) all the syslog feature in Release production?

Do you create messages in textual form (i.e. do you use printf-like 
functions)?

On 07/31/17 08:45, pozz wrote:
On 07/31/17 08:45, pozz wrote:

 >
 > Do you have a facility to disable (not present in the final binary)
 > syslog messages from single modules?

Typically, no, since it can be useful for finding faults and form
the basis of a diagnostic for production and in field testing. If
you work it right, the overhead is minimal. If you want selective
reporting, define another member of the structure for each item to
enable or disable that or define the level, say from none to verbose.
Typically, the structure just carries a U32 count, but can be changed
to include strings etc. if i'm bringing a board up from cold, the
syslog stuff is the first to be built. That then provides built in
diagnostics as you bring up the hal layer for the other hardware and
peripherals.

 >
 > Do you create messages in textual form (i.e. do you use printf-like
 > functions)?

Depends on the system, but typically use as little of the C library as
possible. All the types are wrong etc. Instead, have a well proven
library of simple functions that print strings, convert between number
bases, read input etc. It gets added to over the years to provide
functionality. Of course, text output assumes a uart or other means
of accessing the messages, but the same ideas can be used to
provide flashing led sequences, whatever.

I guess what i'm saying is that I try to build some level of on board
diagnostics into every project from the start, whether or not it's in
the spec. It just makes life so much easier...

Chris