Forums

68332 horrible crash

Started by John Larkin March 9, 2009
Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up
to use another chip select as the autovector generator. This mostly
works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA
to release (raise) the interrupt request pin before it exits.

But sometimes the firmware wants to shut down the associated subsystem
so tells the fpga to clear the interrupt request, which de-asserts
(pulls high) the port F pin. Then the cpu traps with a "spurious
interrupt" error and *everything* is trashed. It takes a full
powerdown/powerup to recover.

I can fix this by never disabling the interrupt signal from the fpga,
and ignoring any unwanted interrupts in the isr. But it's curious.
Seems like requesting the interrupt, then changing your mind, is
really a bad thing to do.

John

John Larkin wrote:
> Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up > to use another chip select as the autovector generator. This mostly > works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA > to release (raise) the interrupt request pin before it exits. > > But sometimes the firmware wants to shut down the associated subsystem > so tells the fpga to clear the interrupt request, which de-asserts > (pulls high) the port F pin. Then the cpu traps with a "spurious > interrupt" error and *everything* is trashed. It takes a full > powerdown/powerup to recover. > > I can fix this by never disabling the interrupt signal from the fpga, > and ignoring any unwanted interrupts in the isr. But it's curious. > Seems like requesting the interrupt, then changing your mind, is > really a bad thing to do. > > John >
You get a "spurious interrupt" if an IRQ is signalled but then cleared before the processor has entered the corresponding interrupt routine. This can happen if the interrupt is signalled for a very short time, especially if you have interrupts disabled for at the time. But there should not be a problem if the firmware responds to an interrupt by telling the FPGA to disable future interrupts. Perhaps the FPGA code has a bug allowing glitches on the interrupt pin when the interrupts are disabled? You could also add a spurious interrupt handler consisting of a single "rte", but that seems a little inelegant. mvh., David
On Mon, 09 Mar 2009 06:48:36 +0100, David Brown
<david.brown@hesbynett.removethisbit.no> wrote:

>John Larkin wrote: >> Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up >> to use another chip select as the autovector generator. This mostly >> works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA >> to release (raise) the interrupt request pin before it exits. >> >> But sometimes the firmware wants to shut down the associated subsystem >> so tells the fpga to clear the interrupt request, which de-asserts >> (pulls high) the port F pin. Then the cpu traps with a "spurious >> interrupt" error and *everything* is trashed. It takes a full >> powerdown/powerup to recover. >> >> I can fix this by never disabling the interrupt signal from the fpga, >> and ignoring any unwanted interrupts in the isr. But it's curious. >> Seems like requesting the interrupt, then changing your mind, is >> really a bad thing to do. >> >> John >> > >You get a "spurious interrupt" if an IRQ is signalled but then cleared >before the processor has entered the corresponding interrupt routine. >This can happen if the interrupt is signalled for a very short time, >especially if you have interrupts disabled for at the time. But there >should not be a problem if the firmware responds to an interrupt by >telling the FPGA to disable future interrupts. > >Perhaps the FPGA code has a bug allowing glitches on the interrupt pin >when the interrupts are disabled? > >You could also add a spurious interrupt handler consisting of a single >"rte", but that seems a little inelegant. > >mvh., > >David
Thanks. My FPGA guy doesn't think he's making glitches, but it sure seems like he is. Sounds like the 68K folks copied the PDP-11 interrupt system, which had a similar "feature" as regards glitch interrupts. I'm an engineer, so I don't mind inelegant. John
John Larkin wrote:
> Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up > to use another chip select as the autovector generator. This mostly > works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA > to release (raise) the interrupt request pin before it exits. > > But sometimes the firmware wants to shut down the associated subsystem > so tells the fpga to clear the interrupt request, which de-asserts > (pulls high) the port F pin. Then the cpu traps with a "spurious > interrupt" error and *everything* is trashed. It takes a full > powerdown/powerup to recover. > > I can fix this by never disabling the interrupt signal from the fpga, > and ignoring any unwanted interrupts in the isr. But it's curious. > Seems like requesting the interrupt, then changing your mind, is > really a bad thing to do. > > John >
When you say "shut down the associated subsystem" do you mean power down a chip that is still driven by another chip that is powered up? If so then this might not be a software bug. John Eaton
John Larkin wrote:
> On Mon, 09 Mar 2009 06:48:36 +0100, David Brown > <david.brown@hesbynett.removethisbit.no> wrote: > >> John Larkin wrote: >>> Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up >>> to use another chip select as the autovector generator. This mostly >>> works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA >>> to release (raise) the interrupt request pin before it exits. >>> >>> But sometimes the firmware wants to shut down the associated subsystem >>> so tells the fpga to clear the interrupt request, which de-asserts >>> (pulls high) the port F pin. Then the cpu traps with a "spurious >>> interrupt" error and *everything* is trashed. It takes a full >>> powerdown/powerup to recover. >>> >>> I can fix this by never disabling the interrupt signal from the fpga, >>> and ignoring any unwanted interrupts in the isr. But it's curious. >>> Seems like requesting the interrupt, then changing your mind, is >>> really a bad thing to do. >>> >>> John >>> >> You get a "spurious interrupt" if an IRQ is signalled but then cleared >> before the processor has entered the corresponding interrupt routine. >> This can happen if the interrupt is signalled for a very short time, >> especially if you have interrupts disabled for at the time. But there >> should not be a problem if the firmware responds to an interrupt by >> telling the FPGA to disable future interrupts. >> >> Perhaps the FPGA code has a bug allowing glitches on the interrupt pin >> when the interrupts are disabled? >> >> You could also add a spurious interrupt handler consisting of a single >> "rte", but that seems a little inelegant. >> >> mvh., >> >> David > > Thanks. My FPGA guy doesn't think he's making glitches, but it sure > seems like he is. > > Sounds like the 68K folks copied the PDP-11 interrupt system, which > had a similar "feature" as regards glitch interrupts. >
Ok, but in this day and age such a "feature" would surely be a candidate for an entry on the errata sheet.
> I'm an engineer, so I don't mind inelegant. >
We all need our kludges :-) -- Regards, Joerg http://www.analogconsultants.com/ "gmail" domain blocked because of excessive spam. Use another domain or send PM.
On Mar 9, 10:15 am, John Larkin
<jjlar...@highNOTlandTHIStechnologyPART.com> wrote:

> Thanks. My FPGA guy doesn't think he's making glitches, but it sure > seems like he is.
Not perfectly reliable, but how about having him put in an interrupt count register, that internally increments from the signal driving that output pin? Then you can ask the FPGA how many interrupts it thinks it's generated (make the register not be cleared by whatever startup procedure you have for the micro, so that it will survive crash & reset) Common causes of FPGA's doing odd things is having an asynchronous input affect more than one thing in the logic, as each part of the fpga may see it through a different time delay or even collection of analog issues. The only thing an external input should really be feeding is an input register - everything else should be based on the registered value, not the raw input. And that output had better be registered too - if it's a combinatorial result its likely to glitch.
On 11 Mar., 21:23, cs_post...@hotmail.com wrote:
> On Mar 9, 10:15 am, John Larkin > > <jjlar...@highNOTlandTHIStechnologyPART.com> wrote: > > Thanks. My FPGA guy doesn't think he's making glitches, but it sure > > seems like he is. > > Not perfectly reliable, but how about having him put in an interrupt > count register, that internally increments from the signal driving > that output pin? =A0Then you can ask the FPGA how many interrupts it > thinks it's generated (make the register not be cleared by whatever > startup procedure you have for the micro, so that it will survive > crash & reset) > > Common causes of FPGA's doing odd things is having an asynchronous > input affect more than one thing in the logic, as each part of the > fpga may see it through a different time delay or even collection of > analog issues. =A0 =A0 The only thing an external input should really be > feeding is an input register - everything else should be based on the > registered value, not the raw input. > > And that output had better be registered too - if it's a combinatorial > result its likely to glitch.
I read it as the CPU crashing not the FPGA, but the likely cause is the same,- a glitch that reaches some logic and not other inside the CPU sending it to lala land. some MCU's latch the interrupt even when you configure it as level triggered, avoiding problem. but it means you have to aknowledge the interrupt twice, in the periperal and in the cpu. guess you could get kinda the same functionality, by -if possible- configuring the interrupt as edge triggered and making the fpga "wiggle" the pin as long as it has an active interrupt. just found this: http://www.freescale.com/files/microcontrollers/doc/app_no= te/M68331_332TUT.pdf 5.2.6 Problem: The Processor Takes a Spurious Interrupt Exception ... 2. There are noise spikes on an IRQ line. Use pull-up resistors on the IRQ lines. 3. A square wave is used to generate external interrupts, and the source of the interrupt is going away before the interrupt is acknowledged. ... An interrupt request signal must remain asserted from the time it first occurs until the end of the IACK cycle. -Lasse
On Mar 11, 7:23 pm, "langw...@fonz.dk" <langw...@fonz.dk> wrote:

> > Common causes of FPGA's doing odd things is having an asynchronous > > input affect more than one thing in the logic
> I read it as the CPU crashing not the FPGA, but the likely cause is > the same,- > a glitch that reaches some logic and not other inside the CPU sending > it to lala land.
If that were the case the CPU is very badly designed. My idea was that such a flaw in the design of the FPGA (user configuration) could get it into unanticipated states and produce glitches/unintended transitions on the interrupt line to the processor.
On Mon, 09 Mar 2009 21:03:32 -0700, John Eaton <nospam@spam.com>
wrote:

>John Larkin wrote: >> Situation: an FPGA drives the IRQ6 input, the portF pin. It's set up >> to use another chip select as the autovector generator. This mostly >> works. The FPGA pulls it low, the ISR runs, and the ISR pokes the FPGA >> to release (raise) the interrupt request pin before it exits. >> >> But sometimes the firmware wants to shut down the associated subsystem >> so tells the fpga to clear the interrupt request, which de-asserts >> (pulls high) the port F pin. Then the cpu traps with a "spurious >> interrupt" error and *everything* is trashed. It takes a full >> powerdown/powerup to recover. >> >> I can fix this by never disabling the interrupt signal from the fpga, >> and ignoring any unwanted interrupts in the isr. But it's curious. >> Seems like requesting the interrupt, then changing your mind, is >> really a bad thing to do. >> >> John >> > >When you say "shut down the associated subsystem" do you mean power >down a chip that is still driven by another chip that is powered up? > >If so then this might not be a software bug. > > >John Eaton
No, what I mean is that, when a particular interrupt-driven scan function isn't running, it seemed logical to disable interrupts. It's basically a device driver that runs in bursts. When it's done, I was turning off the FPGA hardware that generates the IRQ signal. It looks like doing that may create a brief glitch on the IRQ6 signal going into the CPU. But more generally, it's dangrous to disable the interrupt logic in the FPGA if that might happen when IRQ6 is asserted; the 68332 gets upset if you assert an interrupt and then remove it. Only the ISR is allowed to cause the external irq request line to go false. So what I'm doing now is to set a state flag to "done" and, if the ISR gets fired off but the flag is set, it just clears the irq hardware in the fpga and exits. So any extra interrupt requests that happen after we have finished the i/o sequence are allowed to interrupt, but are ignored. There are probably other ways to fix this. Small price to pay for having level-sensitive interrupts. John
On Mar 12, 4:19=A0pm, cs_post...@hotmail.com wrote:
> On Mar 11, 7:23 pm, "langw...@fonz.dk" <langw...@fonz.dk> wrote: > > > > Common causes of FPGA's doing odd things is having an asynchronous > > > input affect more than one thing in the logic > > I read it as the CPU crashing not the FPGA, but the likely cause is > > the same,- > > a glitch that reaches some logic and not other inside the CPU sending > > it to lala land. > > If that were the case the CPU is very badly designed. =A0 My idea was > that such a flaw in the design of the FPGA (user configuration) could > get it into unanticipated states and produce glitches/unintended > transitions on the interrupt line to the processor.
Of course this is not the case, and your counter suggestion was quite appropriate and practical. The CPU32 has been around for almost 2 decades now, and the spurious interrupt vector has predated it by another decade or so on the 68000. It was just that John was hit by a spurious interrupt and the vector had not been setup at all - normally such interrupts do not occur so he has not have had to deal with them. Davids suggestion to just do a RTE in the spurious IRQ handler was something apparently not only I have been doing routinely for ages, it may be ugly but it works OK if the glitches are not too many (and if they are they cannot remain unnoticed). I would guess John has set the handler to a RTE and forgotten about the issue by now :-) . And even if there are no glitches setting the spurious IRQ to a RTE is a good practice, having a single glitch per year on a well designed system is quite unlikely, but not that unthinkable anyway. Dimiter ------------------------------------------------------ Dimiter Popoff Transgalactic Instruments http://www.tgi-sci.com ------------------------------------------------------ http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/