Paul Keinanen wrote:
> <steve@NOSPAMTAfivetrees.com> wrote:
>>TCP is possible (and preferable) so long as there is a failure mechanism 
>>(think "degree of confidence in the link") which allows for failure to *not* 
>>result in a retransmission queue beyond available memory. Makes sense, esp. 
>>with a strictly local link.

> Anyway, the decision, does the customer need 100 % reliability or 100
> % real time performance should be first checked. Usually the customer
> says that both are needed, but in that case the bill will be doubled
> or tripled  due to the extra hardware required (e.g. multiple PCs),
> which usually will return the customer to square one. 
> 
> After the customer really knows what he/she really wants (i.e. is
> ready to pay) can some sensible decisions about the protocol be made.

Paul,

Agreed, the combination of real-time streams and TCP is sketchy, 
especially when the data rate is very high.  UDP is by far the simplest 
approach, but requires a tolerance for loss due to receiver congestion 
or damaged packets.

If lossless delivery is absolutely necessary, TCP is better than rolling 
your own protocol.  But it requires a lot more resources on both ends, 
especially in the sender - several hundred kB of buffer for the sliding 
window, several MB to queue the input stream during congestion periods 
(low throughput), and enough CPU to manage all the buffering and drain 
the queue after a congestion backlog.

The performance of the receiver (or the latency between the devices) 
will also hugely affect the sender's buffer requirements.  The TCP 
Window Scaling option will probably be mandatory, and jumbo Ethernet 
frames will be a great help if supported by all involved.

Richard

On Sun, 21 Aug 2005 01:50:17 +0100, "Steve at fivetrees"
<steve@NOSPAMTAfivetrees.com> wrote:

>Agreed. I said much the same to the customer - who fortunately has vast 
>experience of doing this kind of thing (albeit not from an embedded POV). 
>TCP is possible (and preferable) so long as there is a failure mechanism 
>(think "degree of confidence in the link") which allows for failure to *not* 
>result in a retransmission queue beyond available memory. Makes sense, esp. 
>with a strictly local link.

There seems to be two uses for TCP/IP, to handle corrupted frames and
handle flow control (congestion). With a dedicated point to point link
the frame corruption rate should be quite low. In a point to point
link, the flow control problem is limited to the receiving PC and its
ability _at_all_times_ process the data at the incoming rate.

If the PC can be made to reliably operate at the incoming data rate, I
would suggest dropping the TCP and using UDP instead. I would suggest
just using a simple UDP protocol with a serial number blindly
transmitted from the embedded device.

However, first you should check that the receiving PC and all the
application software is really capable of processing the data at that
rate, even when the user is doing some kind of activities on the PC
(if allowed during the actual capture). Using an other PC, generate
the same rate of frames as the embedded system and check how the
receiving PC performs.   

In many real time applications you do not even expect to always get
100 % of the data, but it is extremely important to design from the
beginning (and not just at a later stage when you discover that some
data is dropped) what to do when the sample is missing (e.g. some
sample specific signal quality indicator). 

For statistical analysis, you could then discard those sampled that
are not OK. 

In a polled system, if a single sample can not be read due to an
_intermittent_ communication failure, there is not much point in
requesting a reread of the value, since the next poll will eventually
bring a fresh value anyway. The only information that you need is that
a sample is missing and do something sensible (e.g. hold the previous
value instead of inserting 0).

In an audio system, if you get a missing sample, it might be a good
idea to get the next few good samples and using some high order (>1)
interpolation to calculate the missing sample. In a video system, it
might be acceptable to repeat an old frame.

In a system aiming for 100 % reliability (such as TCP), the receiver
will have to handle the normal average traffic and in addition to this
also handle any backlogs due to communication failures, thus, the
physical throughput should be well above the average required
throughput to handle the backlog.

Anyway, the decision, does the customer need 100 % reliability or 100
% real time performance should be first checked. Usually the customer
says that both are needed, but in that case the bill will be doubled
or tripled  due to the extra hardware required (e.g. multiple PCs),
which usually will return the customer to square one. 

After the customer really knows what he/she really wants (i.e. is
ready to pay) can some sensible decisions about the protocol be made.

Paul

> If you can afford to loose data, use UDP instead. If not, you will
> have to solve this problem, and I'm honestly interested to hear about
> your aproach.

Yes, UDP is certainly a candidate solution to the aforementioned
problem.  Since the embedded side will be communicating with a local PC
(could literally be a crossover ethernet cable), I'm guessing that the
few dropped packets that potentially could occur with UDP won't be too
much of an issue (and the "end" application can deal with this in my
case).  I would agrue that though simple, the uIP implementation for
UDP would certainly suffice for what I need (i.e., a 200 Megabit/sec
data pipe).  The way folks usually get large amounts of data through
TCP is by playing with the ethernet frame size (making them larger than
1500 bytes...i've seen frame sizes up to 9000 bytes).  Obviously with
only 16K there isn't a ton of room to buffer up packets for TCP.  I may
still give it a go to see what the limits are practically.

As always, I appreciate the input.

Regards,
John Orlando
www.jrobot.net

"Markus Zingg" <m.zingg@nct.ch> wrote in message 
news:o77fg1dmpdg4sepn6rfqefs4a6676ieouo@4ax.com...
>
> Under TCP, you must be able to resend segments up to the so called
> flight size. That said every segment sent out which got not yet acked
> by the oponent can be lost on it's way, hence must be re-sent if the
> need arises. That's why TCP is reliable after all. With a 200MB/sec
> bandwidth the number of not yet acked segments will be impressive. uIP
> is a fine piece of code but it was designed with restricted embedded
> resources in mind and obviousely this is in contrast to the very high
> bandwidth your application requieres. So, unless you can "reproduce"
> your aquiered data there is no way around local buffering. How do you
> want to do this with only 16KB?
>
> If you can afford to loose data, use UDP instead. If not, you will
> have to solve this problem, and I'm honestly interested to hear about
> your aproach.

Agreed. I said much the same to the customer - who fortunately has vast 
experience of doing this kind of thing (albeit not from an embedded POV). 
TCP is possible (and preferable) so long as there is a failure mechanism 
(think "degree of confidence in the link") which allows for failure to *not* 
result in a retransmission queue beyond available memory. Makes sense, esp. 
with a strictly local link.

Steve
http://www.fivetrees.com

"jro" <john@jrobot.net> wrote in message 
news:1124559835.748766.147650@g49g2000cwa.googlegroups.com...
> I, too, have been tasked with a similar project for work recently.  I
> need to be able to stream data that will be produced at a bandwidth in
> excess of 200 Mb/s (from an A/D converter).  This task is primarily a
> transimssion task (minimal reception required; certainly nothing
> high-speed on the reception side).  Essentially, just grab the data,
> packetize it up into a TCP packet, and dump it out ASAP.  The other end
> of the pipe can essentially be thought of as a top of the line PC with
> oodles of RAM/disk storage to support the data coming in.

Not a million miles away from what I'm doing.

> While still evaluating possibilities, my current plan is to use a
> Virtex4 FX12 FPGA which has an embedded PowerPC core already in the
> chip and an integrated tri-mode ethernet MAC hard core also integrated
> into the chip.  Xilinx also has their Gigabit System Reference Design
> (can be found at:

<snip>

Very useful data, John. Thanks.

Steve
http://www.fivetrees.com

"Bryan Hackney" <no@body.home> wrote in message 
news:BrHNe.134134$gL1.5284@tornado.texas.rr.com...
> Steve at fivetrees wrote:
>> I have a new project coming up that calls for a data acquisition board 
>> (wot I have to design) to deliver said data over ethernet at a minimum of 
>> 6.25Mbytes/second (not including packetisation/checksum/TCP/protocol 
>> overheads). I confess to some trepidation.
>>
>
> [...]
>
> Please let me (us) know what you decide on. This is on the edge of 
> experience
> for many if not all small systems developers. Thanks.

Certainly.

Dame fortune has bought me some time - client needs a lower-bandwidth 
ethernet board first. I shall bide my time and research.

Steve
http://www.fivetrees.com

Markus Zingg wrote: (and omitted attribution)
> 
>> This setup uses uIP, a scaled down TCP/IP stack by Adam Dunkels,
>> integrated to run on an UltraController 2 (which is essentially a
>> utilization of the PowerPC core in the Virtex4 that requires no
>> external memory; 16K of program memory and 16K of data memory in the
>> form of BRAM are utilized right on the FPGA fabric to hold code/data).
>> Anyway, the app note shows an example webserver running on the ML403
>> eval board.  I am hoping to play with this over the next few weeks to
>> see what works and what doesn't work with this setup.  I'll report back
>> new stuff as I find it.
> 
> You should not compare a webserver reading mostly static data to your
> data aquisition task with a 200MB/sec bandwidth.
> 
> Under TCP, you must be able to resend segments up to the so called
> flight size. That said every segment sent out which got not yet acked
> by the oponent can be lost on it's way, hence must be re-sent if the
> need arises. That's why TCP is reliable after all. With a 200MB/sec
> bandwidth the number of not yet acked segments will be impressive. uIP
> is a fine piece of code but it was designed with restricted embedded
> resources in mind and obviousely this is in contrast to the very high
> bandwidth your application requieres. So, unless you can "reproduce"
> your aquiered data there is no way around local buffering. How do you
> want to do this with only 16KB?
> 
> If you can afford to loose data, use UDP instead. If not, you will
> have to solve this problem, and I'm honestly interested to hear about
> your aproach.

I have no idea what throughput he needs, but the burst speed of
200MB/S will already be a load. It might be possible to use error
correcting coding with the packets and UDP, assuming he has a
dedicated link.  He should also carefully characterize the burst
length needed and the overall throughput rate.  It may be that the
raw error rate with a dedicated link is small enough to allow
simplification.

-- 
"If you want to post a followup via groups.google.com, don't use
 the broken "Reply" link at the bottom of the article.  Click on 
 "show options" at the top of the article, then click on the 
 "Reply" at the bottom of the article headers." - Keith Thompson

>This setup uses uIP, a scaled down TCP/IP stack by Adam Dunkels,
>integrated to run on an UltraController 2 (which is essentially a
>utilization of the PowerPC core in the Virtex4 that requires no
>external memory; 16K of program memory and 16K of data memory in the
>form of BRAM are utilized right on the FPGA fabric to hold code/data).
>Anyway, the app note shows an example webserver running on the ML403
>eval board.  I am hoping to play with this over the next few weeks to
>see what works and what doesn't work with this setup.  I'll report back
>new stuff as I find it.

You should not compare a webserver reading mostly static data to your
data aquisition task with a 200MB/sec bandwidth.

Under TCP, you must be able to resend segments up to the so called
flight size. That said every segment sent out which got not yet acked
by the oponent can be lost on it's way, hence must be re-sent if the
need arises. That's why TCP is reliable after all. With a 200MB/sec
bandwidth the number of not yet acked segments will be impressive. uIP
is a fine piece of code but it was designed with restricted embedded
resources in mind and obviousely this is in contrast to the very high
bandwidth your application requieres. So, unless you can "reproduce"
your aquiered data there is no way around local buffering. How do you
want to do this with only 16KB?

If you can afford to loose data, use UDP instead. If not, you will
have to solve this problem, and I'm honestly interested to hear about
your aproach.

Markus

Bryan Hackney wrote:
> Steve at fivetrees wrote:
> > I have a new project coming up that calls for a data acquisition board (wot
> > I have to design) to deliver said data over ethernet at a minimum of
> > 6.25Mbytes/second (not including packetisation/checksum/TCP/protocol
> > overheads). I confess to some trepidation.

I, too, have been tasked with a similar project for work recently.  I
need to be able to stream data that will be produced at a bandwidth in
excess of 200 Mb/s (from an A/D converter).  This task is primarily a
transimssion task (minimal reception required; certainly nothing
high-speed on the reception side).  Essentially, just grab the data,
packetize it up into a TCP packet, and dump it out ASAP.  The other end
of the pipe can essentially be thought of as a top of the line PC with
oodles of RAM/disk storage to support the data coming in.

While still evaluating possibilities, my current plan is to use a
Virtex4 FX12 FPGA which has an embedded PowerPC core already in the
chip and an integrated tri-mode ethernet MAC hard core also integrated
into the chip.  Xilinx also has their Gigabit System Reference Design
(can be found at:

http://www.xilinx.com/esp/wired/optical/xlnx_net/gsrd.htm

but it is currently setup to support only a few eval boards (and is a
bit much for what I am looking to do).  However, Xilinx just released
something much closer to my ideal solution in the form of an app-note
called "Minimal Footprint Tri-Mode Ethernet MAC Processing Engine",
which can be found here:

http://direct.xilinx.com/bvdocs/appnotes/xapp807.pdf

This setup uses uIP, a scaled down TCP/IP stack by Adam Dunkels,
integrated to run on an UltraController 2 (which is essentially a
utilization of the PowerPC core in the Virtex4 that requires no
external memory; 16K of program memory and 16K of data memory in the
form of BRAM are utilized right on the FPGA fabric to hold code/data).
Anyway, the app note shows an example webserver running on the ML403
eval board.  I am hoping to play with this over the next few weeks to
see what works and what doesn't work with this setup.  I'll report back
new stuff as I find it.

> Please let me (us) know what you decide on. This is on the edge of experience
> for many if not all small systems developers. Thanks.

Yeah...there definitely isn't a ton of stuff out there right now for
references on gigE in embedded stuff.  Hopefully this will change with
time (and requirements) change.

Regards,
John Orlando
www.jrobot.net