EmbeddedRelated.com
Forums
Memfault State of IoT Report

Emulators Philosophy

Started by Tom September 2, 2005
What are the collective's opinions regarding the use of in-circuit emulators 
for proving safety critical embedded software?

I've heard from some people that they can't imagine how it could be done 
without whereas others have never used an emulator and consider them 
entirely unnecessary. Has JTAG rendered emulators obscelete? 


Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote:

> What are the collective's opinions regarding the use of in-circuit > emulators for proving safety critical embedded software?
Emulators (like simulators) only serve a purpose in testing and debugging, but since actual "proving" cannot ever be done by testing, they are obviously useless for proving. -- Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de) Even if all the snow were burnt, ashes would remain.
"Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message 
news:3nqjg0F2s52gU1@news.dfncis.de...
> Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote: > >> What are the collective's opinions regarding the use of in-circuit >> emulators for proving safety critical embedded software? > > Emulators (like simulators) only serve a purpose in testing and > debugging, but since actual "proving" cannot ever be done by testing, > they are obviously useless for proving. >
I agree that proving cannot be done by testing alone but I do think that testing is the backbone when proving that a system is safe. However, I acknowledge that emulators (and simulators) are only tools to assist in testing, but my question is whether they are essential tools.
Tom wrote:
> "Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message > news:3nqjg0F2s52gU1@news.dfncis.de... > >>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote: >> >>>What are the collective's opinions regarding the use of in-circuit >>>emulators for proving safety critical embedded software? >> >>Emulators (like simulators) only serve a purpose in testing and >>debugging, but since actual "proving" cannot ever be done by testing, >>they are obviously useless for proving. > > I agree that proving cannot be done by testing alone but I do think that > testing is the backbone when proving that a system is safe.
Emulators are not needed for testing. They are helpful for debugging. There are some types of problems that are much more easily debugged with emulators. I worked on one such problem recently: I was getting occasional incorrect handling of a complex event. I didn't have an emulator available. I put debugging outputs on test points to see the realtime sequence of operations. Sometimes a statement in a switch case was being executed when it shouldn't have been. I added realtime trace output to the switch variable just before executing the switch -- the bug went away (Heisenbug!). I removed the trace and added a debug output on a spare pin when that happened. I had another flag that was high during interrupt service. I noticed that the occasional incorrect execution was always occurring exactly 42 us after completion of an interrupt service. I checked an errata sheet and found a problem with the fast interrupt return feature. I then checked the compiler output and saw that it was using the fast return feature. After figuring out how to tell the compiler to not use this feature, the problem went away. It took me 2 days to narrow this down and solve it. With an emulator which has a trace buffer it would have been less, since I could trigger on the symptom and look at the preceding execution. The presence of an emulator would not help me detect the symptom, only fix it.
> However, I > acknowledge that emulators (and simulators) are only tools to assist in > testing, but my question is whether they are essential tools.
Helpful for debugging, yes. Helpful for testing, possibly. Essential, maybe for solving a few types of problems. Thad
> What are the collective's opinions regarding the use of in-circuit emulators > for proving safety critical embedded software?
"Fly what you test and test what you fly". An emulator gives you a realtime view into program state. This is very useful in debugging but not so useful in testing. Testing attempts to prove that the product works as it will be shipped to the customer.
Thad Smith wrote:
<snip>
 >
> I checked an errata sheet and found a problem with > the fast interrupt return feature. I then checked the compiler output > and saw that it was using the fast return feature. After figuring out > how to tell the compiler to not use this feature, the problem went away. > It took me 2 days to narrow this down and solve it. With an emulator > which has a trace buffer it would have been less, since I could trigger > on the symptom and look at the preceding execution. The presence of an > emulator would not help me detect the symptom, only fix it.
... ONLY if the emulator had the SAME silicon flaw that caused the errata. -jg
Tom wrote:

> What are the collective's opinions regarding the use of in-circuit > emulators for proving safety critical embedded software? > > I've heard from some people that they can't imagine how it could be done > without whereas others have never used an emulator and consider them > entirely unnecessary. Has JTAG rendered emulators obscelete?
Emulators for difficult processor chips (those that would otherwise require massive efforts to set up monitoring and trigger points) can be useful at the pre-product-finishing stage when you need to confirm that the system can follow the whole of its programming. However, the final testing must be accomplished with the processors that will be delivered with the system, especially for safety critical systems. This is a phased testing approach and is, in my opinion, quite a valid consideration. I would not expect the testing to be carried out under the emulator alone. I would hope, however, that you would rather shy away from using such a complex and difficult processors for a safety critical system. Where JTAG and other low-level debugging connectivity is offered it can be a great help in the early stages. When some of the software layers are in place, and proven to be working properly, these debugging ports can be left alone. I have found that, for my Forth based systems, I do not need the services of an emulator, hardly a need for the debugging ports at all, as I can gain full control over the hardware I am using. I keep the individual system as simple as is practicable for the application at hand and will distribute over several processors in order to maintain the simplest level, commensurate with the application requirements, in each processing system. -- ******************************************************************** Paul E. Bennett ....................<email://peb@amleth.demon.co.uk> Forth based HIDECS Consultancy .....<http://www.amleth.demon.co.uk/> Mob: +44 (0)7811-639972 Tel: +44 (0)1235-811095 Going Forth Safely ....EBA. http://www.electric-boat-association.org.uk/ ********************************************************************
In article <3nqjg0F2s52gU1@news.dfncis.de>, Hans-Bernhard Broeker
<broeker@physik.rwth-aachen.de> writes
>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote: > >> What are the collective's opinions regarding the use of in-circuit >> emulators for proving safety critical embedded software? > >Emulators (like simulators) only serve a purpose in testing and >debugging, but since actual "proving" cannot ever be done by testing, >they are obviously useless for proving. >
This is completely wrong. There are several SW tools that use full ICE for non-intrusive hard real time unit and system testing. Besides most decent ICE had script languages that permit testing and exercising of code to prove it. More to the point as ICe can feed signals in as well as read them in some cases an ICE is the ONLY way of proving some equipment. -- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ /\/\/ chris@phaedsys.org www.phaedsys.org \/\/\ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
In article <43185f70$0$80022$892e0abb@auth.newsreader.octanews.com>,
Thad Smith <ThadSmith@acm.org> writes
>Tom wrote: >> "Hans-Bernhard Broeker" <broeker@physik.rwth-aachen.de> wrote in message >> news:3nqjg0F2s52gU1@news.dfncis.de... >> >>>Tom <tlucasremoveall@thisextragubbinstoreplyautoflame.co.uk> wrote: >>> >>>>What are the collective's opinions regarding the use of in-circuit >>>>emulators for proving safety critical embedded software? >>> >>>Emulators (like simulators) only serve a purpose in testing and >>>debugging, but since actual "proving" cannot ever be done by testing, >>>they are obviously useless for proving. >> >> I agree that proving cannot be done by testing alone but I do think that >> testing is the backbone when proving that a system is safe. > >Emulators are not needed for testing. They are helpful for debugging. >There are some types of problems that are much more easily debugged with >emulators. I worked on one such problem recently: I was getting >occasional incorrect handling of a complex event. I didn't have an >emulator available. I put debugging outputs on test points to see the >realtime sequence of operations.
This is why ICE are essential. You "put debugging outputs on test points" IE you changed the code..... A good ICE will execute the code without changing it in hard read time. What you are doing it opening the fridge door to check the temperature. You will get a reading but by opening the door you have changed the temperature. For more things the reading is "close enough" but it is not accurate. Changing the code can effect many things including the bug.
>Sometimes a statement in a switch case >was being executed when it shouldn't have been. I added realtime trace >output to the switch variable just before executing the switch -- the >bug went away (Heisenbug!).
DO you mean you use the ICE trace? or is it an example I mentioned above of the test code changing the error?
>interrupt service. I checked an errata sheet and found a problem with
Been there done that :-) Errata sheets are the bane of my life!
> It took me 2 days to narrow this down and solve it. With an emulator >which has a trace buffer it would have been less, since I could trigger >on the symptom and look at the preceding execution. The presence of an >emulator would not help me detect the symptom, only fix it.
You mean would help you find it but not fix it? -- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ /\/\/ chris@phaedsys.org www.phaedsys.org \/\/\ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
In article <4318a5e6$1@clear.net.nz>, Jim Granville
<no.spam@designtools.co.nz> writes
>Thad Smith wrote: ><snip> >> >> I checked an errata sheet and found a problem with >> the fast interrupt return feature. I then checked the compiler output >> and saw that it was using the fast return feature. After figuring out >> how to tell the compiler to not use this feature, the problem went away. >> It took me 2 days to narrow this down and solve it. With an emulator >> which has a trace buffer it would have been less, since I could trigger >> on the symptom and look at the preceding execution. The presence of an >> emulator would not help me detect the symptom, only fix it. > >... ONLY if the emulator had the SAME silicon flaw that caused the errata. > -jg >
Very true.... You need not only to use the same part but the same revision. It depends on the ICE as to how you do this. In some families of MCO you use the same part in the target and the ICE. EG the HOOKS system used on some 8051's -- \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ \/\/\/\/\ Chris Hills Staffs England /\/\/\/\/ /\/\/ chris@phaedsys.org www.phaedsys.org \/\/\ \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

Memfault State of IoT Report