Hi,

I'm working on a proposal to design a box that will control a
scientific gadget. Our box will output frequency sweeps, arbitrary
waveforms, a couple of dozen voltages that can be changed/ramped per
user desires, and some discrete logic levels and triggers.

One architecture would pack an Intel-cpu SBC and a custom board in a
2U rack box. The SBC would talk gigabit ethernet to the customer's
system and PCI to our board.

Something like this, maybe:

http://us.kontron.com/index.php?id=226&cat=527&productid=1726

Our board would have a PCI interface driving a biggish FIFO, say 8k
deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
thing repacks the 32-bit pci transfers into the input of the 48-bit
wide fifo. The output side of the FIFO would be driving a fairly
simple state machine; each fifo word has an opcode field and a data
field, with different opcodes feeding various devices connected to the
physics... dds synthesizers, ttl outputs, whatever. The state machine
that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
we can slow down operations to match the realtime needs of the
experiment and reduce the average fifo feed rate.

OK, we finally get to a question: If we run some flavor of Linux on
the SBC, what's a good strategy for keeping the fifo loaded? Assuming
that we have the recipe for an entire experimental shot in program
ram, some tens of megabytes maybe, we could...

1. Have the fifo logic interrupt the cpu when the fifo is, say, half
empty. The isr would compute how empty the fifo actually is at that
instant and set up a short dma transfer to top it off.

2. A task (or isr) would be run periodically, a thousand times per
second might work, and it would be responsible for topping off the
fifo, either dma or maybe just poking in the data in a loop.

3. Best, if possible: set up a single DMA transfer to do the entire
shot. That involves a dma controller that understands that the target
is sometimes busy, and retries after getting bounced. I know the pci
bus has hooks for split transfers, but I don't know if standard
Intel-type dma controllers can work in this mode.

4. If it's a dual-core cpu, is it hard (under Linux) to assign one cpu
to just do the fifo transfers?

5. Other ideas?

The problem is not unlike feeding a sound card. How does that work?
Maybe we could start with a sound card driver and hack that?

Any suggestions for resources? Books, model drivers, references,
people we could hire to write the drivers for us? Being a hardware
guy, mostly analog at that, I don't know much about this stuff.

Right now, I only have to write up a plausible proposed architecture,
but if we get the job, p>=0.5 maybe, we'll have to do actually it.

Thanks!

John

On Apr 12, 12:38 pm, John Larkin
<jjlar...@highNOTlandTHIStechnologyPART.com> wrote:
> Hi,
>
> I'm working on a proposal to design a box that will control a
> scientific gadget. Our box will output frequency sweeps, arbitrary
> waveforms, a couple of dozen voltages that can be changed/ramped per
> user desires, and some discrete logic levels and triggers.
>
> One architecture would pack an Intel-cpu SBC and a custom board in a
> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> system and PCI to our board.
>
> Something like this, maybe:
>
> http://us.kontron.com/index.php?id=226&cat=527&productid=1726
>
> Our board would have a PCI interface driving a biggish FIFO, say 8k
> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
> thing repacks the 32-bit pci transfers into the input of the 48-bit
> wide fifo. The output side of the FIFO would be driving a fairly
> simple state machine; each fifo word has an opcode field and a data
> field, with different opcodes feeding various devices connected to the
> physics... dds synthesizers, ttl outputs, whatever. The state machine
> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
> we can slow down operations to match the realtime needs of the
> experiment and reduce the average fifo feed rate.

> 5. Other ideas?

I don't see the real need to tightly couple the FIFO to your Intel
computer.  Why not give the FPGA its own memory, either DRAM or Flash
with some "offline" interface to load the entire experiment into it?
You can still put your board in the 2U box with the Intel computer,
but give it a USB port or something similar which wouldn't be too hard
to program up via an MCU (making the MCU and DRAM look like a storage
device should be a good solution).  Or if you have any reason to
separate your board and the Intel computer, you could have the data
transferred via an SD card or similar.

Certainly a PCI interface is not rocket science, but talking to your
board in real time is likely a bit more work than other solutions.  I
did the FPGA design for a similar project to record and playback high
speed data.  We used a DRAM buffer on the card and had to stream data
to/from a RAID hard drive.  The software *was* the long pole in the
tent, even using a real time version of NT.  So speaking from
experience, you might be better off giving the FPGA plenty of RAM/
Flash and a less direct connection to the Intel computer.

Are your experiments likely to need more than a few gig of
instructions?

John Larkin wrote:

> Hi,
> 
> I'm working on a proposal to design a box that will control a
> scientific gadget. Our box will output frequency sweeps, arbitrary
> waveforms, a couple of dozen voltages that can be changed/ramped per
> user desires, and some discrete logic levels and triggers.
> 
> One architecture would pack an Intel-cpu SBC and a custom board in a
> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> system and PCI to our board.
> 
> Something like this, maybe:
> 
> http://us.kontron.com/index.php?id=226&cat=527&productid=1726
> 
> Our board would have a PCI interface driving a biggish FIFO, say 8k
> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
> thing repacks the 32-bit pci transfers into the input of the 48-bit
> wide fifo. The output side of the FIFO would be driving a fairly
> simple state machine; each fifo word has an opcode field and a data
> field, with different opcodes feeding various devices connected to the
> physics... dds synthesizers, ttl outputs, whatever. The state machine
> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
> we can slow down operations to match the realtime needs of the
> experiment and reduce the average fifo feed rate.
> 
> OK, we finally get to a question: If we run some flavor of Linux on
> the SBC, what's a good strategy for keeping the fifo loaded? Assuming
> that we have the recipe for an entire experimental shot in program
> ram, some tens of megabytes maybe, we could...
> 
> 1. Have the fifo logic interrupt the cpu when the fifo is, say, half
> empty. The isr would compute how empty the fifo actually is at that
> instant and set up a short dma transfer to top it off.
> 
> 2. A task (or isr) would be run periodically, a thousand times per
> second might work, and it would be responsible for topping off the
> fifo, either dma or maybe just poking in the data in a loop.
> 
> 3. Best, if possible: set up a single DMA transfer to do the entire
> shot. That involves a dma controller that understands that the target
> is sometimes busy, and retries after getting bounced. I know the pci
> bus has hooks for split transfers, but I don't know if standard
> Intel-type dma controllers can work in this mode.
> 
> 4. If it's a dual-core cpu, is it hard (under Linux) to assign one cpu
> to just do the fifo transfers?
> 
> 5. Other ideas?
> 
> The problem is not unlike feeding a sound card. How does that work?
> Maybe we could start with a sound card driver and hack that?
> 
> Any suggestions for resources? Books, model drivers, references,
> people we could hire to write the drivers for us? Being a hardware
> guy, mostly analog at that, I don't know much about this stuff.
> 
> Right now, I only have to write up a plausible proposed architecture,
> but if we get the job, p>=0.5 maybe, we'll have to do actually it.
> 
> Thanks!
> 
> John
> 
> 
If you want my opinion, I've been down that road using DMA controllers 
and trying to keep up with the technology as it changes..
   I found that having a self contained Buffer on board (stand Alone) is 
the best, using a common link like RJ-45/ Ethernet via a TCP to access
the information from the buffers a much better approach! that is, if you
time critical isn't a problem to synchronize something?
   Using a 1GB or even the 100 Mb links should yield good results..
With that, You can support any platform that has a Ethernet interface.

  if you want to go the route of linux or freeBSD, you can compile those
as embedded operations in which case the full attention of the system 
can stay with your PCI card how ever, I would still use your own On 
board cache. The better sound cards for example do a min of 512 bytes 
and up on their own onboard cache before transporting that to the system 
because other hardware activities can stall the board.

   That is just my opinion of course.

http://webpages.charter.net/jamie_5";

John Larkin wrote:
> Hi,
> 
> I'm working on a proposal to design a box that will control a
> scientific gadget. Our box will output frequency sweeps, arbitrary
> waveforms, a couple of dozen voltages that can be changed/ramped per
> user desires, and some discrete logic levels and triggers.
> 
> One architecture would pack an Intel-cpu SBC and a custom board in a
> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> system and PCI to our board.
> 
> Something like this, maybe:
> 
> http://us.kontron.com/index.php?id=226&cat=527&productid=1726
> 
> Our board would have a PCI interface driving a biggish FIFO, say 8k
> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
> thing repacks the 32-bit pci transfers into the input of the 48-bit
> wide fifo. The output side of the FIFO would be driving a fairly
> simple state machine; each fifo word has an opcode field and a data
> field, with different opcodes feeding various devices connected to the
> physics... dds synthesizers, ttl outputs, whatever. The state machine
> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
> we can slow down operations to match the realtime needs of the
> experiment and reduce the average fifo feed rate.
> 
> OK, we finally get to a question: If we run some flavor of Linux on
> the SBC, what's a good strategy for keeping the fifo loaded? Assuming
> that we have the recipe for an entire experimental shot in program
> ram, some tens of megabytes maybe, we could...
> 
> 1. Have the fifo logic interrupt the cpu when the fifo is, say, half
> empty. The isr would compute how empty the fifo actually is at that
> instant and set up a short dma transfer to top it off.
> 
> 2. A task (or isr) would be run periodically, a thousand times per
> second might work, and it would be responsible for topping off the
> fifo, either dma or maybe just poking in the data in a loop.
> 
> 3. Best, if possible: set up a single DMA transfer to do the entire
> shot. That involves a dma controller that understands that the target
> is sometimes busy, and retries after getting bounced. I know the pci
> bus has hooks for split transfers, but I don't know if standard
> Intel-type dma controllers can work in this mode.
> 
> 4. If it's a dual-core cpu, is it hard (under Linux) to assign one cpu
> to just do the fifo transfers?
> 
> 5. Other ideas?
> 
> The problem is not unlike feeding a sound card. How does that work?
> Maybe we could start with a sound card driver and hack that?
> 
> Any suggestions for resources? Books, model drivers, references,
> people we could hire to write the drivers for us? Being a hardware
> guy, mostly analog at that, I don't know much about this stuff.
> 
> Right now, I only have to write up a plausible proposed architecture,
> but if we get the job, p>=0.5 maybe, we'll have to do actually it.
> 
> Thanks!
> 
> John
> 
> 

You're in effect putting a microprogrammed CPU in that FPGA so what's 
the big deal with it fetching the new FIFO data for itself from an 
external store, and you can design the architecture for that any way you 
want, within the constraints of the OS that is, some kind of dual port 
job. Seems to be better than the micromanaging you propose...

John Larkin wrote:
> ...
>... Our box will output frequency sweeps, arbitrary
> waveforms, a couple of dozen voltages that can be changed/ramped per
> user desires, and some discrete logic levels and triggers.
>
> One architecture would pack an Intel-cpu SBC and a custom board in a
> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> system and PCI to our board.
>
> ...
> ...
>
> 5. Other ideas?
>

Since the echo you are getting so far indicates the latency using a
x86
may well be too high under linux or whatever, I can suggest doing some
tiny
DPS thing for you - with PCI and Ethernet. The latency then is no
issue,
tcp/ip etc. comes with it, filesystem/disk etc. If you settle for 100
MbpS Ethernet,
it is quite easy for me - I can reuse some of the MPC5200 designs I
have.
1 GbpS it will take some other part and more than 3-4 months, though.
I am not sure I can beat the cost&time of someone writing the thing
for you
under linux while I do the whole thing, but I am willing to try hard
to do so,
the time has come when I want to make all that stuff I have more
popular than it is now.

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

Original message: http://groups.google.com/group/comp.arch.embedded/msg/3b61a5c2043b8477?dmode=source

On Sat, 12 Apr 2008 14:12:56 -0400, Fred Bloggs <n...@nospam.com>
wrote:

>
>
>John Larkin wrote:
>> Hi,
>> 
>> I'm working on a proposal to design a box that will control a
>> scientific gadget. Our box will output frequency sweeps, arbitrary
>> waveforms, a couple of dozen voltages that can be changed/ramped per
>> user desires, and some discrete logic levels and triggers.
>> 
>> One architecture would pack an Intel-cpu SBC and a custom board in a
>> 2U rack box. The SBC would talk gigabit ethernet to the customer's
>> system and PCI to our board.
>> 
>> Something like this, maybe:
>> 
>> http://us.kontron.com/index.php?id=226&cat=527&productid=1726
>> 
>> Our board would have a PCI interface driving a biggish FIFO, say 8k
>> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
>> thing repacks the 32-bit pci transfers into the input of the 48-bit
>> wide fifo. The output side of the FIFO would be driving a fairly
>> simple state machine; each fifo word has an opcode field and a data
>> field, with different opcodes feeding various devices connected to the
>> physics... dds synthesizers, ttl outputs, whatever. The state machine
>> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
>> we can slow down operations to match the realtime needs of the
>> experiment and reduce the average fifo feed rate.
>> 
>> OK, we finally get to a question: If we run some flavor of Linux on
>> the SBC, what's a good strategy for keeping the fifo loaded? Assuming
>> that we have the recipe for an entire experimental shot in program
>> ram, some tens of megabytes maybe, we could...
>> 
>> 1. Have the fifo logic interrupt the cpu when the fifo is, say, half
>> empty. The isr would compute how empty the fifo actually is at that
>> instant and set up a short dma transfer to top it off.
>> 
>> 2. A task (or isr) would be run periodically, a thousand times per
>> second might work, and it would be responsible for topping off the
>> fifo, either dma or maybe just poking in the data in a loop.
>> 
>> 3. Best, if possible: set up a single DMA transfer to do the entire
>> shot. That involves a dma controller that understands that the target
>> is sometimes busy, and retries after getting bounced. I know the pci
>> bus has hooks for split transfers, but I don't know if standard
>> Intel-type dma controllers can work in this mode.
>> 
>> 4. If it's a dual-core cpu, is it hard (under Linux) to assign one cpu
>> to just do the fifo transfers?
>> 
>> 5. Other ideas?
>> 
>> The problem is not unlike feeding a sound card. How does that work?
>> Maybe we could start with a sound card driver and hack that?
>> 
>> Any suggestions for resources? Books, model drivers, references,
>> people we could hire to write the drivers for us? Being a hardware
>> guy, mostly analog at that, I don't know much about this stuff.
>> 
>> Right now, I only have to write up a plausible proposed architecture,
>> but if we get the job, p>=0.5 maybe, we'll have to do actually it.
>> 
>> Thanks!
>> 
>> John
>> 
>> 
>
>You're in effect putting a microprogrammed CPU in that FPGA


No, the fpga state machines, the ones that load and unload the fifo,
will be very simple. Ditto the dds synthsizers (which we have already,
from another product) and dac drivers.

>so what's 
>the big deal with it fetching the new FIFO data for itself from an 
>external store, and you can design the architecture for that any way you 
>want, within the constraints of the OS that is, some kind of dual port 
>job. Seems to be better than the micromanaging you propose...

Both using local-to-the-FPGA sequence store (which would have to be
wide DRAM... lots of pins) or having our board bus-master and suck
data out of the CPU ram, are a lot more parts and engineering. And
we'd still need the PCI interface to load the local ram.

The SBC already has a gig of dram, all the refresh stuff, and the DMA
controllers. And two CPUs. Given that we don't expect very high
average transfer rates, probably around a megabyte per second (we need
to understand the physics better and quantify that better) the
architecture I suggested is probably the simplest. The physics pretty
much limits how fast the experiment can ever go, so if we have
reasonable margin we'd be good forever.

We could buy an FPGA pci soft core (or use one of the public ones) or
even just use a PLX chip to handshake the PCI transactions for the
fpga. We're a small company, and can't put a massive project team on
this, so we prefer a simple architecture that has a high probability
of working soon.

John

John Larkin wrote:
> 
> Hi,
> 
> I'm working on a proposal to design a box that will control a
> scientific gadget. Our box will output frequency sweeps, arbitrary
> waveforms, a couple of dozen voltages that can be changed/ramped per
> user desires, and some discrete logic levels and triggers.
> 
> One architecture would pack an Intel-cpu SBC and a custom board in a
> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> system and PCI to our board.
> 
> Something like this, maybe:
> 
> http://us.kontron.com/index.php?id=226&cat=527&productid=1726
> 
> Our board would have a PCI interface driving a biggish FIFO, say 8k
> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
> thing repacks the 32-bit pci transfers into the input of the 48-bit
> wide fifo. The output side of the FIFO would be driving a fairly
> simple state machine; each fifo word has an opcode field and a data
> field, with different opcodes feeding various devices connected to the
> physics... dds synthesizers, ttl outputs, whatever. The state machine
> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
> we can slow down operations to match the realtime needs of the
> experiment and reduce the average fifo feed rate.
> 
> OK, we finally get to a question: If we run some flavor of Linux on
> the SBC, what's a good strategy for keeping the fifo loaded? Assuming
> that we have the recipe for an entire experimental shot in program
> ram, some tens of megabytes maybe, we could...
> 
> 1. Have the fifo logic interrupt the cpu when the fifo is, say, half
> empty. The isr would compute how empty the fifo actually is at that
> instant and set up a short dma transfer to top it off.
> 
> 2. A task (or isr) would be run periodically, a thousand times per
> second might work, and it would be responsible for topping off the
> fifo, either dma or maybe just poking in the data in a loop.

Having the board generate an interrupt and initiate a DMA transfer seems
more elegant than depending on a polled design. 
 
> 3. Best, if possible: set up a single DMA transfer to do the entire
> shot. That involves a dma controller that understands that the target
> is sometimes busy, and retries after getting bounced. I know the pci
> bus has hooks for split transfers, but I don't know if standard
> Intel-type dma controllers can work in this mode.

There are dedicated PCI controllers that have all the smarts needed to
manage DMA transfers. All you'd need is some handshaking between your
FPGA and a PCI controller chip. 

Its been a while since I played with any of this so I don't have part
numbers handy. But it wasn't rocket science 5 years ago.

The other way to do this is to put the PCI controller function (DMA
management, etc.) in your FPGA. Google for 'open cores', 'open source
hardware', etc. and 'PCI controller'. If there's enough room in your
FPGA, it would simplify the board design to do it this way.
 

-- 
Paul Hovnanian	p...@hovnanian.com
-----------------------------------------------------------------------
Procrastinators: The leaders for tomorrow.

On Apr 12, 3:21 pm, John Larkin
<jjlar...@highNOTlandTHIStechnologyPART.com> wrote:
> On Sat, 12 Apr 2008 14:12:56 -0400, Fred Bloggs <nos...@nospam.com>
> wrote:
>
>
>
>
>
> >John Larkin wrote:
> >> Hi,
>
> >> I'm working on a proposal to design a box that will control a
> >> scientific gadget. Our box will output frequency sweeps, arbitrary
> >> waveforms, a couple of dozen voltages that can be changed/ramped per
> >> user desires, and some discrete logic levels and triggers.
>
> >> One architecture would pack an Intel-cpu SBC and a custom board in a
> >> 2U rack box. The SBC would talk gigabit ethernet to the customer's
> >> system and PCI to our board.
>
> >> Something like this, maybe:
>
> >>http://us.kontron.com/index.php?id=226&cat=527&productid=1726
>
> >> Our board would have a PCI interface driving a biggish FIFO, say 8k
> >> deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
> >> thing repacks the 32-bit pci transfers into the input of the 48-bit
> >> wide fifo. The output side of the FIFO would be driving a fairly
> >> simple state machine; each fifo word has an opcode field and a data
> >> field, with different opcodes feeding various devices connected to the
> >> physics... dds synthesizers, ttl outputs, whatever. The state machine
> >> that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
> >> we can slow down operations to match the realtime needs of the
> >> experiment and reduce the average fifo feed rate.
>
> >> OK, we finally get to a question: If we run some flavor of Linux on
> >> the SBC, what's a good strategy for keeping the fifo loaded? Assuming
> >> that we have the recipe for an entire experimental shot in program
> >> ram, some tens of megabytes maybe, we could...
>
> >> 1. Have the fifo logic interrupt the cpu when the fifo is, say, half
> >> empty. The isr would compute how empty the fifo actually is at that
> >> instant and set up a short dma transfer to top it off.
>
> >> 2. A task (or isr) would be run periodically, a thousand times per
> >> second might work, and it would be responsible for topping off the
> >> fifo, either dma or maybe just poking in the data in a loop.
>
> >> 3. Best, if possible: set up a single DMA transfer to do the entire
> >> shot. That involves a dma controller that understands that the target
> >> is sometimes busy, and retries after getting bounced. I know the pci
> >> bus has hooks for split transfers, but I don't know if standard
> >> Intel-type dma controllers can work in this mode.
>
> >> 4. If it's a dual-core cpu, is it hard (under Linux) to assign one cpu
> >> to just do the fifo transfers?
>
> >> 5. Other ideas?
>
> >> The problem is not unlike feeding a sound card. How does that work?
> >> Maybe we could start with a sound card driver and hack that?
>
> >> Any suggestions for resources? Books, model drivers, references,
> >> people we could hire to write the drivers for us? Being a hardware
> >> guy, mostly analog at that, I don't know much about this stuff.
>
> >> Right now, I only have to write up a plausible proposed architecture,
> >> but if we get the job, p>=0.5 maybe, we'll have to do actually it.
>
> >> Thanks!
>
> >> John
>
> >You're in effect putting a microprogrammed CPU in that FPGA
>
> No, the fpga state machines, the ones that load and unload the fifo,
> will be very simple. Ditto the dds synthsizers (which we have already,
> from another product) and dac drivers.
>
> >so what's
> >the big deal with it fetching the new FIFO data for itself from an
> >external store, and you can design the architecture for that any way you
> >want, within the constraints of the OS that is, some kind of dual port
> >job. Seems to be better than the micromanaging you propose...
>
> Both using local-to-the-FPGA sequence store (which would have to be
> wide DRAM... lots of pins) or having our board bus-master and suck
> data out of the CPU ram, are a lot more parts and engineering. And
> we'd still need the PCI interface to load the local ram.
>
> The SBC already has a gig of dram, all the refresh stuff, and the DMA
> controllers. And two CPUs. Given that we don't expect very high
> average transfer rates, probably around a megabyte per second (we need
> to understand the physics better and quantify that better) the
> architecture I suggested is probably the simplest. The physics pretty
> much limits how fast the experiment can ever go, so if we have
> reasonable margin we'd be good forever.

I would again suggest that you can *simplify* the project by putting
RAM on your board instead of a *real time* PCI interface.

You can add a GB of DRAM to your board that with just a very few
chips, I haven't looked at the available RAM chips lately but I
believe they are beyond the Gbit level.  Or you can use a module and
plug in what ever size you need.  You can lose the PCI interface by
using something serial which can be done with a single MCU chip.
Check out the Luminary Micro parts with Ethernet including the PHY.
PCI is not really all that fast and getting any real speed out of it
will take a fair amount of programming effort.

The simple MCU receives the data over USB or Ethernet and fills the
RAM through an interface in the FPGA.  The FPGA reads the RAM as
needed for playback. Your post says you *don't* need high average
transfer rates to the FIFO, so you wouldn't need high transfer rates
from the RAM.  So you could use an 8 bit memory interface and clock it
slowly if you wanted.

This is all very simple stuff to implement and get working.  DRAMs
handle their own refresh if you just give them a strobe periodically.
Your app reads consecutively, so if it reads fast enough, you don't
even need refresh.  If it is not reading fast enough you have plenty
of time to do a refresh.

I have worked with SDRAM in exactly this application before.  If you
are interested in doing this, I can offer help with the FPGA and RAM
design.


> We could buy an FPGA pci soft core (or use one of the public ones) or
> even just use a PLX chip to handshake the PCI transactions for the
> fpga. We're a small company, and can't put a massive project team on
> this, so we prefer a simple architecture that has a high probability
> of working soon.

To think of this as a "massive" effort seems like an overestimate.
Like I said in my other post, I worked on a design very similar to
this and the hard part was in the software.  To make the durn computer
run anything remotely like real time took a lot of effort.  Your
original post did not mention a real time version of Linux which would
almost certainly be required.

Maybe your team is better at the software than hardware.  But
typically a well defined and constrained hardware design effort goes
much better than a software effort that requires the cooperation of a
large, complex operating system.

>One architecture would pack an Intel-cpu SBC and a custom board in a
>2U rack box. The SBC would talk gigabit ethernet to the customer's
>system and PCI to our board.

2U is ugly in that you can't get full height PCI cards without
using a riser kludge to turn the card sideways.

I think PCI cards fit in 3U.  There is a short size that fits in 2U.
(or you can cross your fingers on the riser stuff.)


>Our board would have a PCI interface driving a biggish FIFO, say 8k
>deep by 48 bits wide, inside an FPGA. A simple state machine/latch/mux
>thing repacks the 32-bit pci transfers into the input of the 48-bit
>wide fifo. The output side of the FIFO would be driving a fairly
>simple state machine; each fifo word has an opcode field and a data
>field, with different opcodes feeding various devices connected to the
>physics... dds synthesizers, ttl outputs, whatever. The state machine
>that unloads the fifo would run at 128 MHz, but one opcode is WAIT, so
>we can slow down operations to match the realtime needs of the
>experiment and reduce the average fifo feed rate.

>OK, we finally get to a question: If we run some flavor of Linux on
>the SBC, what's a good strategy for keeping the fifo loaded? Assuming
>that we have the recipe for an entire experimental shot in program
>ram, some tens of megabytes maybe, we could...


>3. Best, if possible: set up a single DMA transfer to do the entire
>shot. That involves a dma controller that understands that the target
>is sometimes busy, and retries after getting bounced. I know the pci
>bus has hooks for split transfers, but I don't know if standard
>Intel-type dma controllers can work in this mode.

I think that's what you want to do.  It comes for free.  I think
it will all make sense if you read the PCI specs.  Or maybe
just the specs for the PCI interface block you are going to use.

Ignoring pipeline problems, the host side of a DMA read request
doesn't know how how much data the device wants.  It just gets
an op-code that says read or read-cache-line.  Once data
starts flowing, either side can say I'm-done-now.  If the device
(still) wants more data, it starts over with bus arbitration.
The host may say "done" to let another device have a turn
or to cross a page boundary or ...


The DMA section of the FPGA will run in chatter mode.  When
there is room in the FIFO for another cache block, it will
ask for more data.  When the FIFO is near-full, it will stop
asking.  You have to leave enough room in the FIFO to hold all
the data in the pipeline.


One quirk.  The driver has to allocate a chunk of physically
contigious memory.  That probably has to happen early in the
boot-up time so you still have a chunk of contigious memory
to grab.

-- 
These are my opinions, not necessarily my employer's.  I hate spam.

>I would again suggest that you can *simplify* the project by putting
>RAM on your board instead of a *real time* PCI interface.

But he doesn't need any real time software.  All he needs
is a big buffer in memory.  A DMA engine on the card will
grab data from memory whenever the FIFO has room.



>You can add a GB of DRAM to your board that with just a very few
>chips, I haven't looked at the available RAM chips lately but I
>believe they are beyond the Gbit level.  Or you can use a module and
>plug in what ever size you need.  You can lose the PCI interface by
>using something serial which can be done with a single MCU chip.
>Check out the Luminary Micro parts with Ethernet including the PHY.
>PCI is not really all that fast and getting any real speed out of it
>will take a fair amount of programming effort.

Yes, a PC is overkill for just grabbing the data and buffering it
for the FPGA.  There may well be better overall designs that don't
use a PC or PCI.

On the other hand, the PCI part of the board design is only ~40 or
~70 wires.  I think the PCI logic is roughly as complicated as the
DRAM interface logic.  (handwave)


-- 
These are my opinions, not necessarily my employer's.  I hate spam.