Emulators Philosophy

What are the collective's opinions regarding the use of in-circuit emulators 
for proving safety critical embedded software?

I've heard from some people that they can't imagine how it could be done 
without whereas others have never used an emulator and consider them 
entirely unnecessary. Has JTAG rendered emulators obscelete? 

Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:

> What are the collective's opinions regarding the use of in-circuit
> emulators for proving safety critical embedded software?

Emulators (like simulators) only serve a purpose in testing and
debugging, but since actual "proving" cannot ever be done by testing,
they are obviously useless for proving.

-- 
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

"Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message 
news:3nqjg0F2s52gU1@news.dfncis.de...
> Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:
>
>> What are the collective's opinions regarding the use of in-circuit
>> emulators for proving safety critical embedded software?
>
> Emulators (like simulators) only serve a purpose in testing and
> debugging, but since actual "proving" cannot ever be done by testing,
> they are obviously useless for proving.
>

I agree that proving cannot be done by testing alone but I do think that 
testing is the backbone when proving that a system is safe. However, I 
acknowledge that emulators (and simulators) are only tools to assist in 
testing, but my question is whether they are essential tools. 

Tom wrote:
> "Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message 
> news:3nqjg0F2s52gU1@news.dfncis.de...
> 
>>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:
>>
>>>What are the collective's opinions regarding the use of in-circuit
>>>emulators for proving safety critical embedded software?
>>
>>Emulators (like simulators) only serve a purpose in testing and
>>debugging, but since actual "proving" cannot ever be done by testing,
>>they are obviously useless for proving.
> 
> I agree that proving cannot be done by testing alone but I do think that 
> testing is the backbone when proving that a system is safe. 

Emulators are not needed for testing.  They are helpful for debugging. 
There are some types of problems that are much more easily debugged with 
emulators.  I worked on one such problem recently:  I was getting 
occasional incorrect handling of a complex event.  I didn't have an 
emulator available.  I put debugging outputs on test points to see the 
realtime sequence of operations.  Sometimes a statement in a switch case 
was being executed when it shouldn't have been.  I added realtime trace 
output to the switch variable just before executing the switch -- the 
bug went away (Heisenbug!).  I removed the trace and added a debug 
output on a spare pin when that happened.  I had another flag that was 
high during interrupt service.  I noticed that the occasional incorrect 
execution was always occurring exactly 42 us after completion of an 
interrupt service.  I checked an errata sheet and found a problem with 
the fast interrupt return feature.  I then checked the compiler output 
and saw that it was using the fast return feature.  After figuring out 
how to tell the compiler to not use this feature, the problem went away. 
  It took me 2 days to narrow this down and solve it.  With an emulator 
which has a trace buffer it would have been less, since I could trigger 
on the symptom and look at the preceding execution.  The presence of an 
emulator would not help me detect the symptom, only fix it.

> However, I 
> acknowledge that emulators (and simulators) are only tools to assist in 
> testing, but my question is whether they are essential tools. 

Helpful for debugging, yes.  Helpful for testing, possibly.  Essential, 
maybe for solving a few types of problems.

Thad

> What are the collective's opinions regarding the use of in-circuit emulators
> for proving safety critical embedded software?

"Fly what you test and test what you fly".

An emulator gives you a realtime view into program state. This is very
useful in debugging but not so useful in testing. Testing attempts to
prove that the product works as it will be shipped to the customer.

Thad Smith wrote:
<snip>
 >
> I checked an errata sheet and found a problem with 
> the fast interrupt return feature.  I then checked the compiler output 
> and saw that it was using the fast return feature.  After figuring out 
> how to tell the compiler to not use this feature, the problem went away. 
>  It took me 2 days to narrow this down and solve it.  With an emulator 
> which has a trace buffer it would have been less, since I could trigger 
> on the symptom and look at the preceding execution.  The presence of an 
> emulator would not help me detect the symptom, only fix it.

... ONLY if the emulator had the SAME silicon flaw that caused the errata.
  -jg

Tom wrote:

> What are the collective's opinions regarding the use of in-circuit
> emulators for proving safety critical embedded software?
> 
> I've heard from some people that they can't imagine how it could be done
> without whereas others have never used an emulator and consider them
> entirely unnecessary. Has JTAG rendered emulators obscelete?

Emulators for difficult processor chips (those that would otherwise require 
massive efforts to set up monitoring and trigger points) can be useful at 
the pre-product-finishing stage when you need to confirm that the system 
can follow the whole of its programming. However, the final testing must be 
accomplished with the processors that will be delivered with the system, 
especially for safety critical systems. This is a phased testing approach 
and is, in my opinion, quite a valid consideration. I would not expect the 
testing to be carried out under the emulator alone. I would hope, however, 
that you would rather shy away from using such a complex and difficult 
processors for a safety critical system.

Where JTAG and other low-level debugging connectivity is offered it can be 
a great help in the early stages. When some of the software layers are in 
place, and proven to be working properly, these debugging ports can be left 
alone.

I have found that, for my Forth based systems, I do not need the services 
of an emulator, hardly a need for the debugging ports at all, as I can gain 
full control over the hardware I am using. I keep the individual system as 
simple as is practicable for the application at hand and will distribute 
over several processors in order to maintain the simplest level, 
commensurate with the application requirements, in each processing system. 

-- 
********************************************************************
Paul E. Bennett ....................<email://peb@amleth.demon.co.uk>
Forth based HIDECS Consultancy .....<http://www.amleth.demon.co.uk/>
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-811095
Going Forth Safely ....EBA. http://www.electric-boat-association.org.uk/
********************************************************************

In article <3nqjg0F2s52gU1@news.dfncis.de>, Hans-Bernhard Broeker
<broeker@physik.rwth-aachen.de> writes
>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:
>
>> What are the collective's opinions regarding the use of in-circuit
>> emulators for proving safety critical embedded software?
>
>Emulators (like simulators) only serve a purpose in testing and
>debugging, but since actual "proving" cannot ever be done by testing,
>they are obviously useless for proving.
>

This is completely wrong. There are several SW tools that use full ICE
for non-intrusive hard real time unit and system testing. 

Besides most decent ICE had script languages that permit testing and
exercising of code to prove it. More to the point as ICe can feed
signals in as well as read them in some cases an ICE is the ONLY way of
proving some equipment. 


-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
/\/\/ chris@phaedsys.org      www.phaedsys.org \/\/\
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In article <43185f70$0$80022$892e0abb@auth.newsreader.octanews.com>,
Thad Smith <ThadSmith@acm.org> writes
>Tom wrote:
>> "Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message 
>> news:3nqjg0F2s52gU1@news.dfncis.de...
>> 
>>>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:
>>>
>>>>What are the collective's opinions regarding the use of in-circuit
>>>>emulators for proving safety critical embedded software?
>>>
>>>Emulators (like simulators) only serve a purpose in testing and
>>>debugging, but since actual "proving" cannot ever be done by testing,
>>>they are obviously useless for proving.
>> 
>> I agree that proving cannot be done by testing alone but I do think that 
>> testing is the backbone when proving that a system is safe. 
>
>Emulators are not needed for testing.  They are helpful for debugging. 
>There are some types of problems that are much more easily debugged with 
>emulators.  I worked on one such problem recently:  I was getting 
>occasional incorrect handling of a complex event.  I didn't have an 
>emulator available.  I put debugging outputs on test points to see the 
>realtime sequence of operations.  


This is why ICE are essential. You "put debugging outputs on test
points" IE you changed the code..... A good ICE will execute the code
without changing it in hard read time. 

What you are doing it opening the fridge door to check the temperature.
You will get a reading but by opening the door you have changed the
temperature. For more things the reading is "close enough" but it is not
accurate. 

Changing the code can effect many things including the bug.

>Sometimes a statement in a switch case 
>was being executed when it shouldn't have been.  I added realtime trace 
>output to the switch variable just before executing the switch -- the 
>bug went away (Heisenbug!).

DO you mean you use the ICE trace? or is it an example I mentioned above
of the test code changing the error? 

>interrupt service.  I checked an errata sheet and found a problem with 

Been there done that :-) Errata sheets are the bane of my life!

> It took me 2 days to narrow this down and solve it.  With an emulator 
>which has a trace buffer it would have been less, since I could trigger 
>on the symptom and look at the preceding execution.  The presence of an 
>emulator would not help me detect the symptom, only fix it.

You mean would help you find it but not fix it? 




-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
/\/\/ chris@phaedsys.org      www.phaedsys.org \/\/\
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In article <4318a5e6$1@clear.net.nz>, Jim Granville
<no.spam@designtools.co.nz> writes
>Thad Smith wrote:
><snip>
>>
>> I checked an errata sheet and found a problem with 
>> the fast interrupt return feature.  I then checked the compiler output 
>> and saw that it was using the fast return feature.  After figuring out 
>> how to tell the compiler to not use this feature, the problem went away. 
>>  It took me 2 days to narrow this down and solve it.  With an emulator 
>> which has a trace buffer it would have been less, since I could trigger 
>> on the symptom and look at the preceding execution.  The presence of an 
>> emulator would not help me detect the symptom, only fix it.
>
>... ONLY if the emulator had the SAME silicon flaw that caused the errata.
> -jg
>
Very true.... You need not only to  use the same part but the same
revision. It depends on the ICE as to how you do this. 

In some families of MCO you use the same part in the target and the ICE.
EG the HOOKS system used on some 8051's

-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
/\/\/ chris@phaedsys.org      www.phaedsys.org \/\/\
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/