Embedded Ethernet and Corporate LAN

We have an embedded system, that has a SMSC 91C111 Ethernet Chip, ARM
Processor, and Windows CE 3.0.  Our system works great when not exposed to a
"Corporate Ethernet" environment.  When we put our system on a DSL line, it
works great.  We get massive buffer overflows on a typical LAN network.
I have been looking for others who have embedded products, that have similar
experiences.  Does anyone develop embedded products, that go onto LAN
environments?  What did you do to reduce "House Keeping" signals (DHCP
OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?

Sorry if this is the incorrect group for this question, but found it on
embedded.com website.

Thank you

TECH_NEWS wrote:
> We get massive buffer overflows on a typical LAN network.
> What did you do to reduce "House Keeping" signals (DHCP
> OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?

* It'd possible that the environment is to blame, but it'd have to be 
really bad.  A "normal" corporate LAN would only see ~10 broadcast 
packets per second.  OTOH, the system should be able to do 10pps in its 
sleep, so maybe it's a combination of environment and code.

* Have you set the broadcast receive flags in the controller?  You 
should have promiscuous mode turned off, along with all types of 
multicasts.  This will cause the controller to pre-filter all the 
incoming packets.

* Is your MAC address loaded into the controller (it becomes the unicast 
address filter)?

* You will need to have broadcasts enabled because some of the broadcast 
protocols (all F's MAC) are critical (e.g., ARP)

* What sequence are you using to validate the received packets for 
processing?  Because many of the broadcast packets will not be useful to 
your application, you need a very streamlined way to discard them.  An 
excellent tip is to do the filtering steps (address, frame type, 
protocol, port) before validating the checksum, since it's very CPU 
intensive.

* Is the driver interrupt-triggered, or does it poll?  If polled, do you 
check for the next packet right after you're finished with the last one? 
  If it's interrupt triggered, is it possible your interrupts aren't 
working as expected under a moderate load?  For example, look at how 
interrupts and flags are being reset - how promptly, possible race 
conditions, etc.

Richard

On 2005-05-27, TECH_NEWS <info@tcdood.com> wrote:

> We have an embedded system, that has a SMSC 91C111 Ethernet
> Chip, ARM Processor, and Windows CE 3.0. Our system works
> great when not exposed to a "Corporate Ethernet" environment.
> When we put our system on a DSL line, it works great.  We get
> massive buffer overflows on a typical LAN network.

Oops.  Sucks to be you. ;)

> I have been looking for others who have embedded products,
> that have similar experiences.  Does anyone develop embedded
> products, that go onto LAN environments?

Sure.  I spent 5 years working with various ARM based boards
that sit on LANs. I never had any problems.

> What did you do to reduce "House Keeping" signals (DHCP
> OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?

You don't "reduce" them -- you use an OS and network stack that
don't suck.  The network stack should discard the packets it
doesn't care about.  Something in your OS or network stack is
severly broken.

Maybe you should call up Microsoft and get some of that
much-vaunted "support" they're always halucinating about when
they're bashing open-source stuff like the embedded OS I used
on my ARM boards.

-- 
Grant Edwards                   grante             Yow!  I am covered with
                                  at               pure vegetable oil and I am
                               visi.com            writing a best seller!

On 2005-05-27, Grant Edwards <grante@visi.com> wrote:

>> I have been looking for others who have embedded products,
>> that have similar experiences.  Does anyone develop embedded
>> products, that go onto LAN environments?
>
> Sure.  I spent 5 years working with various ARM based boards
> that sit on LANs. I never had any problems.
>
>> What did you do to reduce "House Keeping" signals (DHCP
>> OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?
>
> You don't "reduce" them -- you use an OS and network stack that
> don't suck.  The network stack should discard the packets it
> doesn't care about.  Something in your OS or network stack is
> severly broken.

I should mention it could also be the Ethernet controller
driver that's broken.  If it's receiving in promiscuous mode,
the network stack will see more packets than it should. But
even then you're just wasting some CPU time -- the network
stack should be able to discard them more than fast enough to
avoid overruns.

-- 
Grant Edwards                   grante             Yow!  Do you like "TENDER
                                  at               VITTLES"?
                               visi.com            

TECH_NEWS wrote:
> We have an embedded system, that has a SMSC 91C111 Ethernet Chip, ARM
> Processor, and Windows CE 3.0.  Our system works great when not
> exposed to a "Corporate Ethernet" environment.  When we put our
> system on a DSL line, it works great.  We get massive buffer
> overflows on a typical LAN network.
> I have been looking for others who have embedded products, that have
> similar experiences.  Does anyone develop embedded products, that go
> onto LAN environments?  What did you do to reduce "House Keeping"
> signals (DHCP OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?
>
> Sorry if this is the incorrect group for this question, but found it
> on embedded.com website.

Have you used something like Ethereal to see exactly what level of traffic
you are being exposed to?

When you connect your device to a "Corporate Ethernet" do you connect
through a hub or switch? (I'd expect it to be a switch as true hubs are
relatively rare these days.) If it is a switch then its multicast/broadcast
traffic that's hitting your system. With that in mind you should be able to
write a simple broadcast program to hit your box under controlled rate
conditions.

You didn't say whether you had developed the CE 3.0 BSP for your system or
whether it was someone else's. If yours then Richard's response provides a
good plan.

Is it your _application_ on the embedded system that is failing or the
network stack itself?

I have developed a CE3.0 implementation with both a cable modem and an
Ethernet port and the system worked well. The CE3.0 IP stack isn't
brilliant, as intimated by Grant, but provided that the application code is
written correctly it should be fine. I didn't develop the application but,
on "my" system, the application initially "lost" UDP traffic because it
didn't service read requests fast enough. This was fixed by partly rewriting
the application's network code.

    Andrew

Is this a commerical stack product or homegrown?

Things to check:
	1)Ethernet MAC is not in promiscious mode.

	2)Is the system slow or completly dead?

	2a)If it is completly dead is there some function or protcol that triggers dead mode (ethereal helps here)

	2b)If it's slow is it slow because too much CPU time or out of network buffers/resources?

Richar
"Richard H." <rh86@no.spam> wrote in message
news:uhwle.3699$Gp.3625@fed1read04...
> TECH_NEWS wrote:
> > We get massive buffer overflows on a typical LAN network.
> > What did you do to reduce "House Keeping" signals (DHCP
> > OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?
>
> * It'd possible that the environment is to blame, but it'd have to be
> really bad.  A "normal" corporate LAN would only see ~10 broadcast
> packets per second.  OTOH, the system should be able to do 10pps in its
> sleep, so maybe it's a combination of environment and code.
>
> * Have you set the broadcast receive flags in the controller?  You
> should have promiscuous mode turned off, along with all types of
> multicasts.  This will cause the controller to pre-filter all the
> incoming packets.
>
> * Is your MAC address loaded into the controller (it becomes the unicast
> address filter)?
>
> * You will need to have broadcasts enabled because some of the broadcast
> protocols (all F's MAC) are critical (e.g., ARP)
>
> * What sequence are you using to validate the received packets for
> processing?  Because many of the broadcast packets will not be useful to
> your application, you need a very streamlined way to discard them.  An
> excellent tip is to do the filtering steps (address, frame type,
> protocol, port) before validating the checksum, since it's very CPU
> intensive.
>
> * Is the driver interrupt-triggered, or does it poll?  If polled, do you
> check for the next packet right after you're finished with the last one?
>   If it's interrupt triggered, is it possible your interrupts aren't
> working as expected under a moderate load?  For example, look at how
> interrupts and flags are being reset - how promptly, possible race
> conditions, etc.
>
> Richard

Richar
"Richard H." <rh86@no.spam> wrote in message
news:uhwle.3699$Gp.3625@fed1read04...
> TECH_NEWS wrote:
> > We get massive buffer overflows on a typical LAN network.
> > What did you do to reduce "House Keeping" signals (DHCP
> > OFFERS, ARP REQUESTS, DNS, NETBIOS BROADCASTS)?
>
> * It'd possible that the environment is to blame, but it'd have to be
> really bad.  A "normal" corporate LAN would only see ~10 broadcast
> packets per second.  OTOH, the system should be able to do 10pps in its
> sleep, so maybe it's a combination of environment and code.
>
> * Have you set the broadcast receive flags in the controller?  You
> should have promiscuous mode turned off, along with all types of
> multicasts.  This will cause the controller to pre-filter all the
> incoming packets.
>
> * Is your MAC address loaded into the controller (it becomes the unicast
> address filter)?
>
> * You will need to have broadcasts enabled because some of the broadcast
> protocols (all F's MAC) are critical (e.g., ARP)
>
> * What sequence are you using to validate the received packets for
> processing?  Because many of the broadcast packets will not be useful to
> your application, you need a very streamlined way to discard them.  An
> excellent tip is to do the filtering steps (address, frame type,
> protocol, port) before validating the checksum, since it's very CPU
> intensive.
>
> * Is the driver interrupt-triggered, or does it poll?  If polled, do you
> check for the next packet right after you're finished with the last one?
>   If it's interrupt triggered, is it possible your interrupts aren't
> working as expected under a moderate load?  For example, look at how
> interrupts and flags are being reset - how promptly, possible race
> conditions, etc.
>
> Richard