Modern debuggers cause bad code quality

Oliver Betz wrote:
> "Boudewijn Dijkstra" wrote:
>
> [...]
>
>>> Could it be that today's sophisticated tools lead to more "try and
>>> error", less thinking before doing?
>>
>> Don't blame the debuggers, by far the most developers don't even use
>> high-end "sophisticated" debuggers with can make full use of the on-chip
>> debug logic and provide reliable CPU trace.  Today's hardware and software
>
> IMO that's no counter-argument. My point is, that a painful test cycle
> motivates people to think before stupidly trying dozens of changes.
>

Both ends of that sound masochistic. :)

>> are way more complex, however.  Below are some cynical remarks.
>>
>> We use hardware with badly documented and/or broken peripherals, which
>> requires debugging.
>
> Also to me, "hardware debugging" is one of the most important aspects
> of a powerful debugger. Not only bad or broken foreign hardware but
> also own hardware under development. Being able ad-hoc to twiddle with
> my peripheral or front-end without the need of programming a
> dedicatred "interface" is priceless.
>
> Oliver
>

It'll also possibly yield lots of false positive results. Broken
peripherals especially behave differently under load.

-- 
Les Cargill

Oliver Betz wrote:
> rickman wrote:
>
> [...]
>
>>> In the early days of embedded computing, most embedded developers
>>> could use a TTY interface at best and instrumented the code with some
>>> print statements if something went wrong.
>>
>> What do you mean "early days"?  That still works for me.
>
> it's often (not always) inefficient compared to on-chip-debugging.
>
> Oliver
>


Sometimes the overhead of TTY output ... Heisenbugs the system, but
these days just hitting a breakpoint probably means a full system
restart.

I'm biased because most of the bugs I chase these days are in released
product  and all the easy debug strategies have been used and failed.
I spend most of my time developing resources to *reproduce* bugs; some
are pretty obscure.

Debuggers tend to encourage people to run through it once, then mentally 
line through that function.  This, of course, varies. It's
nice to have options.

-- 
Les Cargill

On 12/4/2014 6:00 AM, Jacob Sparre Andersen wrote:
> Don Y wrote:
>>> Don Y wrote:
>
>>>> I would write:
>>>>
>>>>      ASSERT(elapsed_time != 0)
>>>>      average_speed = distance_traveled / elapsed_time;
>
>> In production, assertions become no-ops -- they don't have handlers
>> associated with them (because they are NEVER supposed to be violated)
>
> I only disable assertions if I can't afford not to.
>
> Failed assertion checks are - when possible - logged, and the impacted
> component is restarted.

And you magically hope something has changed so the condition doesn't
perpetuate?  That's called a bug.

> If I have to disable some assertions, I prefer to have a (machine
> generated) proof that the assertion is always true.

Different type of assertion.

static foo(int) {...}

bar(){
    ...

    while (x > 0) {
       foo(x);
       x--;
    }

    ...

    ASSERT(x == 0)
}

would you want a machine to generate that before having confidence in it?

Or, in the above:

foo(int y) {
    ASSERT(y != 0)
    ...
}

Or:

     uint a, b, c;
     ...
     ASSERT(b>0, c>0)
     ...
     a = b/c;
     ASSERT(a <= b)

Code that NEVER executes shouldn't be present in an executable -- even
if its presence there is to prove that it never executes!

On 12/4/2014 6:30 AM, Les Cargill wrote:
> Sometimes the overhead of TTY output ... Heisenbugs the system, but
> these days just hitting a breakpoint probably means a full system
> restart.

That depends on what you are debugging and whether or not the balance
of the system can tolerate pausing the component under test.  In
my systems, the debugger is integrated with the "OS" so that it
effectively suspends the process/task/thread that is being tested.
As such, the rest of the system can be allowed to run -- albeit
"waiting" on the suspended element(s).

> I'm biased because most of the bugs I chase these days are in released
> product  and all the easy debug strategies have been used and failed.
> I spend most of my time developing resources to *reproduce* bugs; some
> are pretty obscure.

Black boxes are cheap (unless you are in a severely resource constrained
environment -- even there, you can glob on extra resources for the BB
that need not be present in production).  I find them invaluable to
troubleshoot real-time activities (where the cost of gathering data
can disrupt the activity that is being profiled).  They are lightweight
and can usually be independently sized/resized as you deem fit.

> Debuggers tend to encourage people to run through it once, then mentally line
> through that function.  This, of course, varies. It's
> nice to have options.

Oliver Betz wrote:

> Tim Wescott wrote:
>>On a related note, I've been taking more and more to test driven
>>development over the last decade, because it seems to help my development
>>a lot.  In the last two years I've gotten much more strict about doing
> 
> I agree that TDD promotes thinking on the problem but it's IMO not the
> key.
 
TDD is OK if you start the testing early enough. By early enough I am 
hinting you begin your initial testing on the resilience and robustness of 
the Requirements Specification. Getting that bit right deals with where 44% 
of the errors are introduced.

-- 
********************************************************************
Paul E. Bennett IEng MIET.....<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy.............<http://www.hidecs.co.uk>
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

Oliver Betz wrote:

> Paul E Bennett wrote:
> 
> [...]
> 
>>> Could it be that today's sophisticated tools lead to more "try and
>>> error", less thinking before doing?
>>
>>Talk about cats amongst pigeons.
> 
> causing the foreseeable defensiveness.
> 
> [...]
> 
>>Errors that creep into projects are quite language and technology
>>agnostic.
> 
> Ganssle presented numbers: 50..100 errors/KLOC in C, 5..10 in ADA,
> zero with SPARK.
> 
> Of course, this doesn't disagree with your next statement:
> 
>>44% of the projects errors will be inserted within the specification stage
>>(See "Out of Control by the UK Health and Safety Executive). This is why
>>it makes sense to remove those errors before you start the design effort.

I think I know some of the places where Jack might have obtained his numbers 
(and I do not necessarily disagree with them). The languages are not used in 
isolation from a development process so you have to look at the overall 
package for a proper comparison.

If you look at the development environments where each of those languages 
are used you will find that the reason for the differences are more to do 
with the development process users of those languages go through. The SPARK 
guys usually mathematically prove a lot about the Requirements Specification 
before they get to coding (usually auto-coding from within their modelling 
and proof tools).

If those who used the other languages applied a stronger development process 
the numbers across the selection of languages would not be that different. I 
would quote the Les Hatton paper on that if I could remember which one it 
was.

I have already outlined my own development process and considerations in a 
previous post and this stands me in quite good stead. I programme in 
Assembler, Forth and the IEC-61131 set for the most part (but have also code 
some systems in D3 and S80 as well. Don't worry if you haven't heard of the 
last two, they were rather specialised).

The biggest benefit you will get is by making your process deal with 
Requirements Specification Improvements before you get to design and coding. 
That is tackling where the 44% of problems originate.

-- 
********************************************************************
Paul E. Bennett IEng MIET.....<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy.............<http://www.hidecs.co.uk>
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

Don Y wrote:
> On 12/4/2014 6:00 AM, Jacob Sparre Andersen wrote:

>> Failed assertion checks are - when possible - logged, and the
>> impacted component is restarted.
>
> And you magically hope something has changed so the condition doesn't
> perpetuate?  That's called a bug.

Yes.  Letting a component continue to run in a known wrong state is not
acceptable, as that may lead to spreading of the fault.

(And yes, there a of course a lot of "buts" for this.)

> Different type of assertion.
>
> static foo(int) {...}
>
> bar(){
>    ...
>
>    while (x > 0) {
>       foo(x);
>       x--;
>    }
>
>    ...
>
>    ASSERT(x == 0)
> }
>
> would you want a machine to generate that before having confidence in
> it?

Yes - as I haven't seen what "..." does and how "x" is declared.  If "x"
is an integer type, and "..." doesn't touch "x", then a static analyzer
should be able to prove the assertion for you.

> Or, in the above:
>
> foo(int y) {
>    ASSERT(y != 0)
>    ...
> }

Here I would have to know that "foo" is only called from the shown loop.
As I typically program in Ada or SPARK, I would have written the
specification of "foo" as

   subtype Non_Zero_Integer is Integer
     with Static_Predicate => Non_Zero_Integer /= 0;

   procedure Foo (Y : in Non_Zero_Integer);

and have the condition made explicit.  An Ada compiler may or may not
insert a check that "X" is different from 0 before the call to "Foo",
but the SPARK tools would notice that "X" can't be 0 inside the loop and
allow the (unchecked) call to "Foo".

>     uint a, b, c;
>     ...
>     ASSERT(b>0, c>0)
>     ...
>     a = b/c;
>     ASSERT(a <= b)

Yes.  Provable (assuming "..." doesn't mess stuff up).

> Code that NEVER executes shouldn't be present in an executable -- even
> if its presence there is to prove that it never executes!

A nice ideal, but I prefer code that shouldn't execute to systems that
fail because "shouldn't" isn't the same as "can't ever".

I know perfectly well that for DO-178B, you can't have dead code in a
system, but I certainly hope that you have valid confirmation that every
single disabled assertion can't ever fail - for EVERY release of the
system.

Using machine generated/checked proofs makes validating the checking
much simpler.  That is why I prefer machine generated/checked proofs to
the manual kind.

Greetings,

Jacob
--
WTFM: Write the Fine Manual

On Wed, 03 Dec 2014 11:12:26 -0700, Don Y <this@is.not.me.com> wrote:

>On 12/3/2014 4:24 AM, Paul E Bennett wrote:
>
>> Have to look after your health to keep doing this enjoyable work
>> for many more years to come.

Sometime in the future, medicine will discover that the proper diet
for a long healthy life is sugared bacon with coffee.

><shrug>  *Something* is gonna get us -- all!

I have restraining orders from both destinations preventing me from
approaching the gates.  Since I have no place to go, I figure that
means I'll have to live forever 8-)

George

On Thu, 04 Dec 2014 11:50:04 +0100, Oliver Betz wrote:

> Tim Wescott wrote:
> 
> [...]
> 
>>> Could it be that today's sophisticated tools lead to more "try and
>>> error", less thinking before doing?
>>
>>Are you thumping your cane on the floor as you complain about kids these
>>days, and how the world is degenerating into crap because of them?
> 
> no, although age helps developing discipline, it's no guarantee, and I
> think older developers are also affected.
> 
> It's tempting to try changes if it is ostensibly quicker. And if you
> know that a cycle for a new try is painful, you get external motivation
> (compensation for a lack of intrinsic motivation).
> 
>>I think if you look, you'll probably find similar complaints in the
>>Bible,
>>or perhaps written in hieroglyphics on stelae in Egypt someplace.
>>
>>Times change.  The details of the screwups change.  The nature of the
>>thinking leading to the screwups doesn't change, nor does the basic
>>solutions.
> 
> Any good example supporting this?

Alas, not really -- just personal observation.  The people that screw up 
are the people who are willing to settle back and declare success at the 
first splashy "it works" run of the device, and then ignore all the 
various details that might trip things up.

(And note -- I'm not just talking software here.  Lazy engineering is lazy 
engineering, although the symptoms are different among the disciplines.  
We all know what buggy code does.  Bad analog circuit design leads to 
devices that don't work when it's cold or hot or wet or really dry, or 
darker than a lab, or brighter than a lab, etc.  Bad mechanical design 
leads to parts that wear out way too fast, or that only sometimes work, or 
that need excessive work on the manufacturing floor to assemble, etc.

>>On a related note, I've been taking more and more to test driven
>>development over the last decade, because it seems to help my
>>development a lot.  In the last two years I've gotten much more strict
>>about doing
> 
> I agree that TDD promotes thinking on the problem but it's IMO not the
> key.

I look at TDD as a means to stay on the right track -- but just going 
through the motions of TDD without any attention to the underlying goal 
means that you're making bugs with more letters attached.

So it's not TDD by itself that helps me -- what mostly helps is the 
process of breaking my development down into tiny increments, and 
verifying each incremental change.  TDD makes me pay more attention to 
keeping the increments tiny, and helps me to resist the temptation to 
write slews of code for expediency's sake.  Having the built-in regression 
test, however, has saved me large amounts of effort from time to time, by 
finding bugs that would have been monstrously hard to dig out with a 
debugger, but that immediately pop up the next build cycle when using TDD.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

On 12/4/2014 7:31 AM, Jacob Sparre Andersen wrote:
> Don Y wrote:
>> On 12/4/2014 6:00 AM, Jacob Sparre Andersen wrote:
>
>>> Failed assertion checks are - when possible - logged, and the
>>> impacted component is restarted.
>>
>> And you magically hope something has changed so the condition doesn't
>> perpetuate?  That's called a bug.
>
> Yes.  Letting a component continue to run in a known wrong state is not
> acceptable, as that may lead to spreading of the fault.

This is like the developer who sees his code "misbehave" and, because
he can't EASILY reproduce it nor identify an obvious cause, decides
that it's "just a fluke" (implying that it will never happen again...
"alpha particle" mentality) and dismisses the observation.

> (And yes, there a of course a lot of "buts" for this.)
>
>> Different type of assertion.
>>
>> static foo(int) {...}
>>
>> bar(){
>>     ...
>>
>>     while (x > 0) {
>>        foo(x);
>>        x--;
>>     }
>>
>>     ...
>>
>>     ASSERT(x == 0)
>> }
>>
>> would you want a machine to generate that before having confidence in
>> it?
>
> Yes - as I haven't seen what "..." does and how "x" is declared.  If "x"
> is an integer type, and "..." doesn't touch "x", then a static analyzer
> should be able to prove the assertion for you.

The point was to suggest that it is NOT (directly) referenced in either
of those elided regions.  But, there may be lots of code there that makes
it tedious for a developer to determine those things -- WITHOUT SOMEONE
HAVING PREVIOUSLY MADE THAT COMMITMENT/DECLARATION for them.

>> Or, in the above:
>>
>> foo(int y) {
>>     ASSERT(y != 0)
>>     ...
>> }
>
> Here I would have to know that "foo" is only called from the shown loop.
> As I typically program in Ada or SPARK, I would have written the
> specification of "foo" as
>
>     subtype Non_Zero_Integer is Integer
>       with Static_Predicate => Non_Zero_Integer /= 0;
>
>     procedure Foo (Y : in Non_Zero_Integer);
>
> and have the condition made explicit.  An Ada compiler may or may not
> insert a check that "X" is different from 0 before the call to "Foo",
> but the SPARK tools would notice that "X" can't be 0 inside the loop and
> allow the (unchecked) call to "Foo".

Great!  When we all have those tools running on EVERY MCU that we'll
ever encounter, life will be grand!  And David's microwave oven will
come with a 12" touchscreen (with rear facing camera to peek into the
cooking chamber so the space occupied by the display can also mimic
a traditional "window").

But, we don't have those tools.  We often can't even do floating point
math, work in protected address spaces, etc.

And, I wouldn't be willing to spend real consumer dollars to afford all
of that hardware in every product that has benefitted from MCU's.

>>      uint a, b, c;
>>      ...
>>      ASSERT(b>0, c>0)
>>      ...
>>      a = b/c;
>>      ASSERT(a <= b)
>
> Yes.  Provable (assuming "..." doesn't mess stuff up).
>
>> Code that NEVER executes shouldn't be present in an executable -- even
>> if its presence there is to prove that it never executes!
>
> A nice ideal, but I prefer code that shouldn't execute to systems that
> fail because "shouldn't" isn't the same as "can't ever".

If you can't *handle* the failed assertion in a manner appropriate for
*it's* (particular) instance, then you've now added another layer of
complexity to your system:  what are the effects of ANY of these
assertions failing and causing the thread/process/application to be restarted
(and "only" making a note of that fact)?

main() {
      ...
      ASSERT(foo);
      ...
      dispense_medication();
      ...
      ASSERT(something);
      ...
      monitor_vital_signs();
      ...
      ASSERT(other);
      ...
}

clearly, the same recovery method for each of the three assertions will
result in three different sets of side-effects.  The implication is that
"dispense_medication()" being executed multiple times is Not A Good Thing.

So, you're better off either crashing in the assertion *or* plowing straight
through AS IF it had not failed (because you made sure of this AT TEST).

If you're going to *handle* a condition that you expected to be invariant,
then write the LIVE code to handle it.  And, be willing to explain why taht
code *is* live (i.e., why have you allowed this condition to go unchecked
to this point?)

> I know perfectly well that for DO-178B, you can't have dead code in a
> system, but I certainly hope that you have valid confirmation that every
> single disabled assertion can't ever fail - for EVERY release of the
> system.

That's part of the job.  That's why you invest in specifications.  And,
good test suites.  It's why you don't shrug off an OBSERVED malfunction
as a "fluke" and assume it won't happen again, etc.

> Using machine generated/checked proofs makes validating the checking
> much simpler.  That is why I prefer machine generated/checked proofs to
> the manual kind.

I will be interested to see how you fare in an environment where that isn't
available to you.  Perhaps a new thread, "Machine generated proofs cause bad
code quality" -- for the same reasons "modern debuggers" are alleged to!  :>