How much RAM needed for low end Ethernet application?

On Wed, 02 Mar 2005 20:01:20 +0000, Adam Dunkels <adam@sics.nospam.se>
wrote:

>The device was connected using RS232 and SLIP. If I remember
>correctly the entire system used around 40k ROM, of which some 20k
>constituted the Java VNC client and the web pages. uIP also is a general
>purpose stack and not a stripped-down application specific stack.

Nice.

>I would suspect that part of the explanation for the differing resource
>requirements between uIP and PowerNet are that throughput is likely to be
>much higher with PowerNet, and that PowerNet has a full socket API that is
>easier to program than the uIP API and that makes porting Unix
>applications straightforward. Also, the PowerNet web server is much more
>advanced than the uIP web server.

PowerNet was designed from the outset for 32 bit CPUs and
plenty of RAM. Yes, it has a BSD style API. All servers, e.g.
Telnet and HTTP share a common architecture based on one
thread per connection. This makes it rather RAM hungry but
it is extremely easy to program. Now that single-chip
micros have 64k RAM, the RAM hunger is much less of an
issue, but a RAM reduction exercise is under way.

Stephen


--
Stephen Pelc, stephenXXX@INVALID.mpeltd.demon.co.uk
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeltd.demon.co.uk - free VFX Forth downloads

Ulf Samuelsson wrote:
> Doesnt help you in a single chip micro.
> Then you have x amount of SRAM regardless
> of where it is.

I'd expect so; I haven't had the luxury of toying with one yet.  The 
Freescale 10/100 integrated looks interesting, but that's probably not 
the one you had in mind. :-)


> If you have to do a resend, then you have to assemble the packet again
> (on the single chip micro)
> This is very likely a performance bottle neck as well.
> 
> The applications I am thinking of is mostly remote supervision/control
> so pages would not be static.

Yes.  The impact will vary by application, of course.  Even with a 
control application, if you can easily re-extract the variable data to 
re-build the missing packet on-the-fly it can be treated much the same 
as static data, perhaps with little more overhead than re-copying the 
packet from buffer to the NIC.  It's the temporal stuff that you really 
have no choice but to buffer.

No question, the easy solution is more resources and large TX buffers to 
handle re-TX entirely within the TCP module.  Or, conversely, limit the 
number of concurrent sessions and outstanding packets (perhaps adjusting 
the # outstanding packets per session on-the-fly, inverse to the # of 
sessions running).

Adam Dunkels wrote:
> How many Ethernet controllers are able to address this memory directly?

Good question.  I've been focused on an NE2000/Asix driver.  While those 
chips use DMA for block writes, it does seem possible to go back and 
re-write a single byte.  A lot of setup for a single I/O, so inefficient 
for truely random accesses, but seems viable to go back and re-write a 
checksum after the packet is buffered.

> I wrote a device driver for the CS8900a and I seem to remember that it
> only provided a sequential interface to its packet buffer (i.e., read and
> write bytes from the start of the packet to the end, no "random" access).

Interesting.  I'll watch for this.  The NE2000 works similarly, but you 
can specify the start address during setup.

> Hmmm, maybe I don't get it, but aren't you placing the entire TCP
> connection state in the Ethernet controller's SRAM then?

Sorry... by 'SRAM' I'm referring to the MCU's SRAM (say, 200 bytes for 
10 sessions).  Keep the stateful variables in MCU SRAM for performance, 
and construct the packets directly in the Ethernet controller's TX 
buffer to avoid double-buffering and MCU RAM overhead.


> Yes, that is a nice trick and saves four bytes of per-connection state.
> However, if I remember correctly, his stack did not do retransmissions but
> relied on the remote host to do retransmissions. This actually works in
> some situations, where both hosts are sending data simultaneously (like an
> HTTP GET / response situation) but does not work for unidirectional
> transfers.

Hmmm... I'm not sure how that could be viable - the sending host would 
need to have an ACK timeout facility.  ISTR from the sequence number of 
the lowest ACK his scheme could identify the file and offset to re-fetch 
for re-TX.  I'll have to put that one back on the reading pile for a 
refresher. :-)


> Hmmm, I don't think it can be extended *that* easily. One problem arises
> when doing retransmissions: there is no way of knowing what sequence
> number to retransmit. Unless you store it in RAM, but that sort of defeats
> the purpose of the trick :-)

Well, I'm questioning my memory now, but I thought the general scheme 
was something like the high-order bits of the sequence # identified the 
file number, and the low-order bits mapped to the file offset.  Clearly 
in violation of rules about ISN randomization, but if a packet goes 
missing, take the Seq# in highest ACK received, go back to that file and 
start at that offset to re-TX.

It'd seem you could dump an entire window's worth of packets on the 
wire, and TX the next packet based on the highest ACK received - either 
the next in sequence, or the "missing" packet.  (A little more 
complexity required, but not much for static content, I think.) 
Clearly, a very specialized scheme, but MCUs tend toward that kind of 
optimization, and it might let you keep a lot of packet outstanding with 
very few resources.

Paul Keinanen <keinanen@sci.fi> writes:

> >And buffering requirements.  If there's only room for one frame
> >then it's very easy to miss incoming frames.
> 
> This is only a problem if the ethernet controller is in the
> promiscuous mode or there are a lot of ethernet broadcasts or you are
> running some unappropriate protocol for such small environments.

You're right.  I'm used to systems that need promiscuous mode
or multicast addresses.

-- 
Darin Johnson
    Laziness is the father of invention

Stephen Pelc wrote:
> On Wed, 2 Mar 2005 19:55:22 +0100, "Ulf Samuelsson"
> <ulf@a-t-m-e-l.com> wrote:
>
>> The question is really related to the specification of the AT91SAM7X
>> processor which has integrated Ethernet.
>>
>> There is no SRAM in the "Ethernet controller".
>> The Ethernet MAC is integrated and shares the memory with
>> the CPU, so there is no extra memory available.
>> Also there is no external bus, so it is really important
>> that enough SRAM memory is in the chip to start with,
>> because this is all that is available.
>
> I see no reason apart from performance why PowerNet should
> not fit. Is an EVB available yet? I could be tempted.

No, this is not an announcemen and don't hold your breath.

I am trying to figure out if we need  more than 64 kB in the part or not.
My conclusion so far is that 64 kB is OK for small systems, but not
for "fullblown" systems.

> Stephen

-- 
Best Regards,
Ulf Samuelsson
ulf@a-t-m-e-l.com
This message is intended to be my own personal view and it
may or may not be shared by my employer Atmel Nordic AB

On Thu, 3 Mar 2005 00:00:03 +0100, "Ulf Samuelsson"
<ulf@a-t-m-e-l.com> wrote:

>Eric Smith wrote:
>> "Ulf Samuelsson" <ulf@a-t-m-e-l.com> writes:
>>> The question is really related to the specification of the AT91SAM7X
>>> processor which has integrated Ethernet.


Does it have an External Bus Interface?

How much RAM on board?

regards,
Johnny.

>The question is really related to the specification of the AT91SAM7X
>processor which has integrated Ethernet.
>
>There is no SRAM in the "Ethernet controller".
>The Ethernet MAC is integrated and shares the memory with
>the CPU, so there is no extra memory available.
>Also there is no external bus, so it is really important
>that enough SRAM memory is in the chip to start with,
>because this is all that is available.
>
>Thanks for comments!

I would be carefull when it comes to the tricks mentioned by some
posters here. Don't forget that there are SYN flood attacks, tear drop
attacks and other malicious activity out there in the internet waiting
to hit your implementation. You must handle these kind of things
reasonably well in a comercial product or otherwise you will have
lot's of problems.

Another issue is how many systems out there simply do not play by the
standards. We see systems anouncing MSS but then do not accept
segments of this size etc. etc. So again, if this is for a heavily
comercial kind of application you must consider these things too.
Especially the SYN flood thing requieres implementing SYN cookies or
other kind of counter measures or else the limitted resources of an
embedded system are eaten up instantly. The customer surely will blame
you if this happens - and not the attacker.

In the TCP stack I implemented, I'm using 50 buffers of 1500 bytes in
size in a pool. There is a low water mark which if reached means that
new connection requests are no longer granted up until enough buffers
become available. In other words there is no fixed relation between
connection and buffers but instead they are dynamically assigned whre
needed. This allows to handle ~20 connections very efficiently, but
also means at the same time that peaks of 70 or more paralell
connections still work reasonably well. The per connection state
overhead is about 50 bytes more per connection. You could easily
tailor the RAM useage by defining a different buffer pool size.

The code size used to implement this is 34 KB. 

The stack supports the following features / protocols:

- ARP
- ICMP
- IP (fragment reasembly supported)
- DHCP client
- UDP
- DNS resolver
- TCP/IP
- SYN cookies
- configurable window size 

I hope above information gives you an idea of what's needed RAM/Flash
size wise.

HTH

Markus

Markus Zingg wrote:
> Especially the SYN flood thing requieres implementing SYN cookies or
> other kind of counter measures or else the limitted resources of an
> embedded system are eaten up instantly.

Markus,

What approach did you implement for SYN cookies?

I've seen a couple schemes - one crypto-based, the other a scheme for 
encoding the setup vars in the ISN.  Both seem to have challenges, not 
the least of which is the limited field size of the Seq# to work with 
(never mind that it violates the requirement for ISN randomization).

For example, did you sacrifice the options header / certain MSS 
increments in the process?  IIRC, this is the one big point of 
compromise to fit in the available field space, with the being 0 RAM 
used until the reply SYN-ACK arrives (i.e., fairly impervious to SYN 
flooding).

Richard

On Thu, 03 Mar 2005 14:35:24 +0100, Markus Zingg wrote:

>>The question is really related to the specification of the AT91SAM7X
>>processor which has integrated Ethernet.
>>
>>There is no SRAM in the "Ethernet controller". The Ethernet MAC is
>>integrated and shares the memory with the CPU, so there is no extra
>>memory available. Also there is no external bus, so it is really
>>important that enough SRAM memory is in the chip to start with, because
>>this is all that is available.
>>
>>Thanks for comments!
> 
> I would be carefull when it comes to the tricks mentioned by some posters
> here. Don't forget that there are SYN flood attacks, tear drop attacks and
> other malicious activity out there in the internet waiting to hit your
> implementation. You must handle these kind of things reasonably well in a
> comercial product or otherwise you will have lot's of problems.

I'm not sure what you mean with "the tricks mentioned by some posters"
(perhaps you could be a little more specific?) but I would also like to
stress the importance of doing heavy testing of a TCP/IP stack. Send as
much "bad" packets to your stack as possible - others surely will.

> Another issue is how many systems out there simply do not play by the
> standards. We see systems anouncing MSS but then do not accept segments
> of this size etc. etc. So again, if this is for a heavily comercial kind
> of application you must consider these things too. 

Failure to adhere to the size of the MSS seems to be the most common way
to diverge from the RFC standards. This may work fine when talking to a
PC, but may result in communication failure when trying to talk to a small
embedded device. Lacking support for IP fragment reassembly also seems
quite common. Some stacks also leave out TCP's congestion control, which
may even lead to network overload. I fully agree with you: compliance with
RFCs is important. 

For a more thorough discussion of RFC compliance of small TCP/IP
implementations, please refer to my MOBISYS'2003 paper "Full TCP/IP for
8-bit architectures", available at
http://www.sics.se/~adam/mobisys2003.pdf

Regards,

/adam
-- 
Adam Dunkels, Swedish Institute of Computer Science
http://www.sics.se/~adam/, <adam@sics.nospam.se>

Ulf Samuelsson wrote about an unannounced AT91SAM7x part with
ethernet:
> I am trying to figure out if we need  more than 64 kB in the part or not.
> My conclusion so far is that 64 kB is OK for small systems, but not
> for "fullblown" systems.

Of RAM?  Yes, that's plenty for small-to-medium size applications.  I'd
be pretty happy with 32KB.  If you want to support largish applications,
you should either include 128KB of RAM, or allow external RAM to be
added.

If you're talking Flash, I'd recommend at least 128KB, though 64KB is
enough for small systems, and 32KB is enough for tiny systems.

> No, this is not an announcemen and don't hold your breath.

If you haven't yet decided the memory sizes, it's obviously quite
a way out yet.

Or if you have already taped out a part with 64KB and are trying to
decide whether to introduce it as a product, the answer is yes.  :-)

Eric