rickman wrote: [...]>>>> A scriptable debugger enables automated tests in the target hardware. >>> >>> Tests that are often built into the MCU as a selftest. It's not just >>> the factory where such tests are performed. >> >> I'm writing about code testing / verification. > >Then why do you need "automated" tests? That says production to me.http://en.wikipedia.org/wiki/Software_testing#Automated_testing Or Google for "automated code testing" or "automated software test". Oliver -- Oliver Betz, Munich http://oliverbetz.de/
Modern debuggers cause bad code quality
Started by ●December 2, 2014
Reply by ●December 11, 20142014-12-11
Reply by ●December 12, 20142014-12-12
On Tue, 09 Dec 2014 13:12:42 -0500, rickman wrote:> On 12/9/2014 1:04 PM, Oliver Betz wrote: >> rickman wrote:<< snip >>>>>> A scriptable debugger enables automated tests in the target hardware. >>> >>> Tests that are often built into the MCU as a selftest. It's not just >>> the factory where such tests are performed. >> >> I'm writing about code testing / verification. > > Then why do you need "automated" tests? That says production to me.You need automated tests for regression testing. As you do software changes, it's a very wise idea to run the whole test suite on the software. You want to do it as early and as often as possible, to make sure that some change you made in part "A" in the software to enable an improvement in part "B" didn't break part "G" or "P" or "Z". -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●December 12, 20142014-12-12
On 12/12/14 07:15, Tim Wescott wrote:> On Tue, 09 Dec 2014 13:12:42 -0500, rickman wrote: > >> On 12/9/2014 1:04 PM, Oliver Betz wrote: >>> rickman wrote: > > << snip >> > >>>>> A scriptable debugger enables automated tests in the target hardware. >>>> >>>> Tests that are often built into the MCU as a selftest. It's not just >>>> the factory where such tests are performed. >>> >>> I'm writing about code testing / verification. >> >> Then why do you need "automated" tests? That says production to me. > > You need automated tests for regression testing. As you do software > changes, it's a very wise idea to run the whole test suite on the > software. You want to do it as early and as often as possible, to make > sure that some change you made in part "A" in the software to enable an > improvement in part "B" didn't break part "G" or "P" or "Z".They can also, /to some extent/, be used to ensure that as the implementation progresses the detailed "structural" implementation corresponds to the high-level "behavioural" specification. In other words they can aid verifying the implementation matches the validated specification.
Reply by ●December 12, 20142014-12-12
On Fri, 12 Dec 2014 11:25:22 +0000, Tom Gardner wrote:> On 12/12/14 07:15, Tim Wescott wrote: >> On Tue, 09 Dec 2014 13:12:42 -0500, rickman wrote: >> >>> On 12/9/2014 1:04 PM, Oliver Betz wrote: >>>> rickman wrote: >> >> << snip >> >> >>>>>> A scriptable debugger enables automated tests in the target >>>>>> hardware. >>>>> >>>>> Tests that are often built into the MCU as a selftest. It's not >>>>> just the factory where such tests are performed. >>>> >>>> I'm writing about code testing / verification. >>> >>> Then why do you need "automated" tests? That says production to me. >> >> You need automated tests for regression testing. As you do software >> changes, it's a very wise idea to run the whole test suite on the >> software. You want to do it as early and as often as possible, to make >> sure that some change you made in part "A" in the software to enable an >> improvement in part "B" didn't break part "G" or "P" or "Z". > > They can also, /to some extent/, be used to ensure that as the > implementation progresses the detailed "structural" implementation > corresponds to the high-level "behavioural" specification. In other > words they can aid verifying the implementation matches the validated > specification.Yes, agreed. However this sort of thing doesn't necessarily need to be automated -- in theory it just needs to be done once. The reason you want to automate is because relentless testing is a Really Good Thing (even if it's neither necessary nor sufficient to good quality), particularly when it comes to catching the kind of problem I mention above. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●December 12, 20142014-12-12
Le 12/12/2014 19:24, Tim Wescott a écrit :> On Fri, 12 Dec 2014 11:25:22 +0000, Tom Gardner wrote: > >> On 12/12/14 07:15, Tim Wescott wrote: >>> On Tue, 09 Dec 2014 13:12:42 -0500, rickman wrote: >>> >>>> On 12/9/2014 1:04 PM, Oliver Betz wrote: >>>>> rickman wrote: >>> >>> << snip >> >>> >>>>>>> A scriptable debugger enables automated tests in the target >>>>>>> hardware. >>>>>> >>>>>> Tests that are often built into the MCU as a selftest. It's not >>>>>> just the factory where such tests are performed. >>>>> >>>>> I'm writing about code testing / verification. >>>> >>>> Then why do you need "automated" tests? That says production to me. >>> >>> You need automated tests for regression testing. As you do software >>> changes, it's a very wise idea to run the whole test suite on the >>> software. You want to do it as early and as often as possible, to make >>> sure that some change you made in part "A" in the software to enable an >>> improvement in part "B" didn't break part "G" or "P" or "Z". >> >> They can also, /to some extent/, be used to ensure that as the >> implementation progresses the detailed "structural" implementation >> corresponds to the high-level "behavioural" specification. In other >> words they can aid verifying the implementation matches the validated >> specification. > > Yes, agreed. However this sort of thing doesn't necessarily need to be > automated -- in theory it just needs to be done once.Unless you prepare to make modifications and you want to automate the testing for the sake of simplicity and of systematicity. If the tests are not automated you or somebody else will take the short path and will only test the barely minimum.> > The reason you want to automate is because relentless testing is a Really > Good Thing (even if it's neither necessary nor sufficient to good > quality), particularly when it comes to catching the kind of problem I > mention above. >
Reply by ●December 12, 20142014-12-12
On Wednesday, December 10, 2014 7:02:53 PM UTC-5, Paul Rubin wrote:> Ed Prochak <edprochak@gmail.com> writes: > > yes that may mean turning off most optimizations when I want the > > compiler to emit machine code the way I wrote it in C code. So I have > > control. > > Turning off optimizations doesn't make undefined behavior defined. You > have to either use compiler flags like -wrapv to specify the behavior > that you want, or else write the code to avoid undefined cases.Yes I agree. That is a given. I assume a programmer that is skilled in the language, including knowing undefined behavior and how to avoid it.
Reply by ●December 12, 20142014-12-12
On Fri, 12 Dec 2014 19:33:06 +0100, Lanarcam wrote:> Le 12/12/2014 19:24, Tim Wescott a écrit : >> On Fri, 12 Dec 2014 11:25:22 +0000, Tom Gardner wrote: >> >>> On 12/12/14 07:15, Tim Wescott wrote: >>>> On Tue, 09 Dec 2014 13:12:42 -0500, rickman wrote: >>>> >>>>> On 12/9/2014 1:04 PM, Oliver Betz wrote: >>>>>> rickman wrote: >>>> >>>> << snip >> >>>> >>>>>>>> A scriptable debugger enables automated tests in the target >>>>>>>> hardware. >>>>>>> >>>>>>> Tests that are often built into the MCU as a selftest. It's not >>>>>>> just the factory where such tests are performed. >>>>>> >>>>>> I'm writing about code testing / verification. >>>>> >>>>> Then why do you need "automated" tests? That says production to me. >>>> >>>> You need automated tests for regression testing. As you do software >>>> changes, it's a very wise idea to run the whole test suite on the >>>> software. You want to do it as early and as often as possible, to >>>> make sure that some change you made in part "A" in the software to >>>> enable an improvement in part "B" didn't break part "G" or "P" or >>>> "Z". >>> >>> They can also, /to some extent/, be used to ensure that as the >>> implementation progresses the detailed "structural" implementation >>> corresponds to the high-level "behavioural" specification. In other >>> words they can aid verifying the implementation matches the validated >>> specification. >> >> Yes, agreed. However this sort of thing doesn't necessarily need to be >> automated -- in theory it just needs to be done once. > > Unless you prepare to make modifications and you want to automate the > testing for the sake of simplicity and of systematicity.That would be why I said "in theory".> If the tests are not automated you or somebody else will take the short > path and will only test the barely minimum.That would be why I wrote the statement below:>> The reason you want to automate is because relentless testing is a >> Really Good Thing (even if it's neither necessary nor sufficient to >> good quality), particularly when it comes to catching the kind of >> problem I mention above. >>-- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●December 12, 20142014-12-12
Ed Prochak <edprochak@gmail.com> writes:> Yes I agree. That is a given. I assume a programmer that is skilled in > the language, including knowing undefined behavior and how to avoid it.You were proposing turning off compiler optimizations in order to make the program do predictable things when it encountered undefined behavior. That's different from avoiding undefined behavior. Avoiding undefined behavior in complex C programs seems to be very difficult for human programmers including experts; thus the interest among some of us in better-defined languages such as Ada.
