Inter 'Board' Communication

I've a 'Main' Board that needs to communicate with 4 other processors on
other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
an A/D-Bus aswell.
My first idea was to take a quad UART an put it on the 'Main' board.
Problem is that they are all expensive. Any other idea?

	   
					
---------------------------------------		
Posted through http://www.EmbeddedRelated.com

On Mon, 12 Apr 2010 06:33:10 -0500, "azraiyl"
<azraiyl@n_o_s_p_a_m.gmail.com> wrote:

>I've a 'Main' Board that needs to communicate with 4 other processors on
>other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
>on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
>an A/D-Bus aswell.
>My first idea was to take a quad UART an put it on the 'Main' board.
>Problem is that they are all expensive. Any other idea?

The satellite processors are on separate boards. How far are they
separated from the main board and are they in the same enclosure?

Is the data rate requirement for a sustained, full-duplex between each
satellite processor and the main?

LVDS comes to mind as a plausible physical layer.

-- 
Rich Webb     Norfolk, VA

azraiyl wrote:
> I've a 'Main' Board that needs to communicate with 4 other processors on
> other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
> on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
> an A/D-Bus aswell.
> My first idea was to take a quad UART an put it on the 'Main' board.
> Problem is that they are all expensive. Any other idea?

Are you expecting 1MB/s as your total throughput?
Or, for each "link"?   (i.e., as if star configured)

Do the other processors ever have to talk to each other
directly?  Or, is everything Master-slave with the
Master always the "Main Board"?

Why can't you use the SPI/I2C channels?  Can you
rethink how you are encoding your data to use lower
data rates?

You could use USB host controller on the Main (easy
to find MCU's with slave controllers built in).

You could use *SCSI* controllers ("SCSI-1" will give you
5MB/s easily).  Not sure if you can find any legacy
products (e.g., 5380) for this  :-(  This also gives you
some structure to the data stream (UARTS don't -- you
have to impose structure on the "character stream")

Likewise, you can use ethernet and probably strip out the
magnetics with some passive trickery (since it is a real
short haul and you know *exactly* how many nodes you have).

You can also use an "analog" physical layer (drive a D/AC
output to all nodes' A/DC inputs).  But, this requires a lot
of processing (unless you deal with everything in the time
domain and keep your noise floor way down).

Personally, I'd explore the USB route, first.  You might be able
to spec different processors (if it is early enough in the
design stage) for your "slaves" and then just take the
hit for an external host controller on the master (there
are some MCUs with this ability but nowhere near as many
as peripherals).

>The satellite processors are on separate boards. How far are they
>separated from the main board and are they in the same enclosure?

Less than 50 cm. More something like extensions boards.

>Is the data rate requirement for a sustained, full-duplex between each
>satellite processor and the main?

Full-Duplex: Yes. Sustained: Depends on the extensions boards. Some have
only 115200 Bits/s others have upto 1 MiB/s.

>LVDS comes to mind as a plausible physical layer.

They are close enough. I currently don't think LVDS is needed.	   
					
---------------------------------------		
Posted through http://www.EmbeddedRelated.com

>Are you expecting 1MB/s as your total throughput?
>Or, for each "link"?   (i.e., as if star configured)

Optimally for each "link".

>Do the other processors ever have to talk to each other
>directly?  Or, is everything Master-slave with the
>Master always the "Main Board"?

Most often only Master/Slave but there should be a possibility for
asynchronous replies.

>Why can't you use the SPI/I2C channels?  Can you
>rethink how you are encoding your data to use lower
>data rates?

Unfortunately the are all used on the processor. The main processor is
currently an OMAP-L138. SPI, UART, I2C, McBSP: Everything is already used.
Ethernet is not available (because of pinmultiplexing with McBSP).

>You could use USB host controller on the Main (easy
>to find MCU's with slave controllers built in).

That's an idea. The processor has an USB port, I only need a USB hub.
Software overhead on the "slave" processors is maybe a bit high.

>You could use *SCSI* controllers ("SCSI-1" will give you
>5MB/s easily).  Not sure if you can find any legacy
>products (e.g., 5380) for this  :-(  This also gives you
>some structure to the data stream (UARTS don't -- you
>have to impose structure on the "character stream")

Noted.

>Likewise, you can use ethernet and probably strip out the
>magnetics with some passive trickery (since it is a real
>short haul and you know *exactly* how many nodes you have).

Ethernet is not availble for the current main processor (see above).

>Personally, I'd explore the USB route, first.  You might be able
>to spec different processors (if it is early enough in the
>design stage) for your "slaves" and then just take the
>hit for an external host controller on the master (there
>are some MCUs with this ability but nowhere near as many
>as peripherals).

I'm early in the design phase and therefore open to anything.	   
					
---------------------------------------		
Posted through http://www.EmbeddedRelated.com

azraiyl wrote:
>> Are you expecting 1MB/s as your total throughput?
>> Or, for each "link"?   (i.e., as if star configured)
> 
> Optimally for each "link".

This could be a potential problem for USB (1.1), then
(store-and-forward USB*2* hub??)

>> Do the other processors ever have to talk to each other
>> directly?  Or, is everything Master-slave with the
>> Master always the "Main Board"?
> 
> Most often only Master/Slave but there should be a possibility for
> asynchronous replies.

But, you can support asynchronous replies within a M-S protocol.
I.e., if said protocol causes each slave to be polled regularly
then the slave could queue a deferred reply for "next polling
time".

>> Why can't you use the SPI/I2C channels?  Can you
>> rethink how you are encoding your data to use lower
>> data rates?
> 
> Unfortunately the are all used on the processor. The main processor is
> currently an OMAP-L138. SPI, UART, I2C, McBSP: Everything is already used.
> Ethernet is not available (because of pinmultiplexing with McBSP).
> 
>> You could use USB host controller on the Main (easy
>> to find MCU's with slave controllers built in).
> 
> That's an idea. The processor has an USB port, I only need a USB hub.
> Software overhead on the "slave" processors is maybe a bit high.

Cheat.  You "know" what you are talking to.  Use that to
trim overhead (certain types of error handling) from the
protocol.

Decouple the processes to a greater extent.  I.e., *don't*
expect synchronous messaging.  Throw a message to a slave
and "move on"; check back later for the reply -- even if
the actual "virtual protocol" remains synchronous.

>> You could use *SCSI* controllers ("SCSI-1" will give you
>> 5MB/s easily).  Not sure if you can find any legacy
>> products (e.g., 5380) for this  :-(  This also gives you
>> some structure to the data stream (UARTS don't -- you
>> have to impose structure on the "character stream")
> 
> Noted.
> 
>> Likewise, you can use ethernet and probably strip out the
>> magnetics with some passive trickery (since it is a real
>> short haul and you know *exactly* how many nodes you have).
> 
> Ethernet is not availble for the current main processor (see above).
> 
>> Personally, I'd explore the USB route, first.  You might be able
>> to spec different processors (if it is early enough in the
>> design stage) for your "slaves" and then just take the
>> hit for an external host controller on the master (there
>> are some MCUs with this ability but nowhere near as many
>> as peripherals).
> 
> I'm early in the design phase and therefore open to anything.	   

How sensitive is the data?  Can you afford to drop any?
Can you afford to delay any?  Can you afford out-of-order
packet handling?  etc.

I.e., are you just looking for a NUMA approach to shared
memory?  Or, something more "abstract"?

In article <l9CdnSXxQYlrmV7WnZ2dnUVZ_uadnZ2d@giganews.com>, 
azraiyl@n_o_s_p_a_m.gmail.com says...
> I've a 'Main' Board that needs to communicate with 4 other processors on
> other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
> on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
> an A/D-Bus aswell.
> My first idea was to take a quad UART an put it on the 'Main' board.
> Problem is that they are all expensive. Any other idea?
> 
> 	   
1MByte/S  is a bit fast for standard async comms.  I use the UART ports
on MSP430 chips to communicate between ADC boards and a storage system
at 115KBaud.

You might try SPI comms.  However,  clocking at 8MHZ may stress some
processors in the slave mode.


Mark Borgerson

 

azraiyl wrote:

> I've a 'Main' Board that needs to communicate with 4 other processors on
> other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
> on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
> an A/D-Bus aswell.
> My first idea was to take a quad UART an put it on the 'Main' board.
> Problem is that they are all expensive. Any other idea?

For that kind of transfer rates, you will need hardware support for 
buffering regardless of the transport used. You can certainly stretch 
UART or SPI or I2C to that speed, however that could result in major 
inconveniences. The standard solution could be Ethernet or PCIe or some 
sort of parallel bus with DMA access.


Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

In article <CJ-dnZCbCpDi917WnZ2dnUVZ_j6dnZ2d@giganews.com>, 
nospam@nowhere.com says...
> 
> 
> azraiyl wrote:
> 
> > I've a 'Main' Board that needs to communicate with 4 other processors on
> > other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
> > on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
> > an A/D-Bus aswell.
> > My first idea was to take a quad UART an put it on the 'Main' board.
> > Problem is that they are all expensive. Any other idea?
> 
> For that kind of transfer rates, you will need hardware support for 
> buffering regardless of the transport used. You can certainly stretch 
> UART or SPI or I2C to that speed, however that could result in major 
> inconveniences. The standard solution could be Ethernet or PCIe or some 
> sort of parallel bus with DMA access.
> 
On the MSP430, I use the UART ports with DMA receive.  Since packet
lengths are fixed, the DMA setup is fairly simple.  The main board
prompts for a packet, sets up the DMA for a receive that puts 
the packet into a queue.  The end-of-packet interrupt increments
the qeueue pointer and count.  If you bumped this up to 1MB/s
the DMA might start taking an appreciable part of the CPU
bandwidth on a 16MHz CPU.   At that data rate, I'd probably
switch to an ARM or Cortex at 30 to 60Mhz so I'd have time
to do something useful with the data.

SPI is a bit trickier, since it requires a transmit DMA channel
to generate the clocks, in addition to the receive DMA channel.

Mark Borgerson

On 12/04/2010 20:28, Mark Borgerson wrote:
> In article<l9CdnSXxQYlrmV7WnZ2dnUVZ_uadnZ2d@giganews.com>,
> azraiyl@n_o_s_p_a_m.gmail.com says...
>> I've a 'Main' Board that needs to communicate with 4 other processors on
>> other boards with at least 1 MiByte/s. A prefer to use the Adress/Data-Bus
>> on the 'Main' Board. The 'other' processors support SPI, I2C, UART and have
>> an A/D-Bus aswell.
>> My first idea was to take a quad UART an put it on the 'Main' board.
>> Problem is that they are all expensive. Any other idea?
>>
>> 	
> 1MByte/S  is a bit fast for standard async comms.  I use the UART ports
> on MSP430 chips to communicate between ADC boards and a storage system
> at 115KBaud.
>

It's uncommon to have speeds about perhaps 921.6 kbaud with RS-232, but 
there are high-speed RS-485 drivers that work happily with 30+ MBaud. 
But it's not going to be easy to get a microcontroller's UART to run at 
the 10 MBaud you need for 1 MB/s - it would have to be external uarts.

> You might try SPI comms.  However,  clocking at 8MHZ may stress some
> processors in the slave mode.
>
>
> Mark Borgerson
>
>