SmartFusion2 SoC

On 16.10.2015 г. 07:30, rickman wrote:
> On 10/15/2015 10:30 PM, Dimiter_Popoff wrote:
>> On 16.10.2015 г. 04:50, rickman wrote:
>>> On 10/15/2015 9:43 PM, Dimiter_Popoff wrote:
>>>> On 16.10.2015 г. 04:19, Les Cargill wrote:
>>>>> Dimiter_Popoff wrote:
>>>>>> On 15.10.2015 г. 07:13, Roberto Waltman wrote:
>>>>>>> Dimiter_Popoff wrote:
>>>>>>>
>>>>>>>> generally you cannot serve webpages using udp. It gives you just a
>>>>>>>> "best effort" delivery, even on a local ethernet packets do get
>>>>>>>> lost.
>>>>>>>
>>>>>>> Rick, Dimiter, check this:
>>>>>>>
>>>>>>> ENet - Reliable UDP networking library
>>>>>>>
>>>>>>> ENet's purpose is to provide a relatively thin, simple and robust
>>>>>>> network communication layer on top of UDP (User Datagram Protocol).
>>>>>>>
>>>>>>> The primary feature it provides is optional reliable, in-order
>>>>>>> delivery of packets.
>>>>>>>
>>>>>>> http://enet.bespin.org/
>>>>>>>
>>>>>>> R.W.
>>>>>>>
>>>>>>
>>>>>> Hi Roberto,
>>>>>>
>>>>>> I gave it a brief glance, looks interesting and is probably useful
>>>>>> for
>>>>>> certain tasks but it is far from a replacement for tcp. From what I
>>>>>> saw
>>>>>> in the overview I believe they ensure in order delivery by waiting
>>>>>> for
>>>>>> the ack before sending a new packet; meaning an RTT between each two
>>>>>> packets, which may mean seconds.
>>>>>> TCP is very well thought through,it can hardly be simplified any
>>>>>> further
>>>>>> and stay useful. For data streaming it is just "the" choice by a
>>>>>> great
>>>>>> margin. It will take a while until all the tiny MCUs are capable
>>>>>> of it but they are closing in on that. For example my tcp/ip
>>>>>> implementation for DPS can easily fit in half a megabyte of code,
>>>>>> that including the entire OS basics (file system, window support, all
>>>>>> the system calls etc., all that uncompromised and not really planned
>>>>>> as "small footprint").
>>>>>>
>>>>>> Dimiter
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------
>>>>>> Dimiter Popoff, TGI             http://www.tgi-sci.com
>>>>>> ------------------------------------------------------
>>>>>> http://www.flickr.com/photos/didi_tgi/
>>>>>>
>>>>>
>>>>> Tactically, UDP is preferable for real-time-ish things.
>>>>> This is especially true if the things are local to each other -
>>>>> say, no the same basically layer 2 network without much routing
>>>>> in place.
>>>>>
>>>>> You don't improve the channel entropy by using TCP; all
>>>>> you do is buy "insurance".
>>>>  > ...
>>>>
>>>> It is not just that. TCP gives you byte level streaming/sequencing out
>>>> of the box, with UDP you have to reinvent that - and you will be
>>>> seriously challenged to match what tcp gives you.
>>>> So UDP is good only when you either do not need that or when you have
>>>> some compelling reason to reinvent it.
>>>
>>> I don't know what you guys are discussing, but the OP's app (mine) would
>>> be something that was highly reliable.
>>>
>>
>> I am speaking not about reliability here, just about what is practical.
>> Considering udp as a transport for http or ftp is just impractical, no
>> sane reason why you would go this way.
>> Obviously if you are chasing reliability you will have to go into all
>> the details yourself and make your pick, the reliability of tcp depends
>> on error detection/correction underneath it (tcp itself only has
>> a checksum per segment, it is up to ip and below (mostly the ethernet
>> CRC or the PPP CRC) to deliver the packets error free - losing some is
>> OK, delivering a damaged one much less so).
>
> Why are you so worried about "reliability" in this one instance while
> nothing else in your computer gives any consideration to errors other
> than possibly the data internal to a hard or flash drive which uses
> somewhat unreliable media.  Other than possibly servers, even RAM no
> longer uses parity which only flags that there was an error, without
> fixing it.  Why do I need anything special to look for errors from a USB
> device?
>

Oh, you probably do not need to do anything for a USB device to correct
errors etc., of course.

Dimiter

rickman <gnuarm@gmail.com> writes:
> Why do I need anything special to look for errors from a USB device?

Do I understand what you're trying to do here?  You want to control some
kind of embedded gadget with a web browser--this is fine.  The gadget
has an ethernet port so you want it to to expose web server directly to
your LAN: good idea in principle.  It means any computer on your LAN can
browse the gadget without any client app.  Very convenient.  The
embedded thing has limited memory so you don't want to put in a full
tcp/ip stack.  In particular you it to support UDP but not TCP.  This is
strained to begin with, but it also defeats the purpose of getting rid
of the client app, since browsers speak TCP.  And the embedded web
application and httpd will consume some resources even without TCP.

So I think the UDP scheme doesn't help you that much.  Easiest may be to
put a dead simple text interface in the gadget (you like Forth so a
Forth interactive session would work nicely), and have a host-side app
(assuming you have something like an rPi talking to the gadget) that
interacts with the gadget and presents an HTTP server and web
application to the LAN.

Is this just about the limitations of that package I linked earlier,
that appeared to have UDP but not TCP?  There has to be TCP support for
it, or in some alternative package.  Really, they would not have put an
ethernet port on that board if they thought running TCP/IP on it was
impractical.  It is fine.

On 16.10.2015 г. 07:09, Les Cargill wrote:
> .....
>>>
>>> You don't improve the channel entropy by using TCP; all
>>> you do is buy "insurance".
>>  > ...
>>
>> It is not just that. TCP gives you byte level streaming/sequencing out
>> of the box, with UDP you have to reinvent that - and you will be
>> seriously challenged to match what tcp gives you.
>
> It's interesting because a lot of things we think of as inherently
> streaming - like VoIP, or internet radio  - are very very much UDP.
> You also get multicast, which can be a big deal.

Multicast can be a big deal of course. And, like I said, there can be
compelling reasons to not use tcp. VOIP and video streaming can well be
among them, tcp will only be in the way for skipping over lost link
periods.
But for say file transfers or sort of telnet interactions you can
only buy yourself more work to do by ditching tcp.
IOW they both have their place - and exist for a reason.


> TCP is fine for what it is. But you still have to have a "dead man"
> timeout  because you won't get a socket closed error reliably when
> you want it.
>
>> So UDP is good only when you either do not need that or when you have
>> some compelling reason to reinvent it.
>>
>
> 99.99% of what I have ever done involved essentially packetized data
> anyway. So that accounts for some of my bias. I don't
> always. but they could all run over UDP, TCP or a serial port. It'd
> just be changing some options and probably adjusting a few things in
> test.
>
> I'd only used UDP in production starting about 2012 or so.

Well I am not biased at all, when I did my tcp/ip stack for DPS I did
implement both, you know :-).

On the timeout for a hanging connection you mentioned elsewhere  - that
is a must of course. I am a lot more aggressive with that, I think the
default value I give for that is 30 seconds or so. You can set it on
a per connection basis, too. Send data out with no ack for that time
and you just get a "timeout error". Putting a receive timeout in the
stack is impractical so the app can check for new data periodically
and eventually time out as needed. It takes just one call to check
how many bytes there are available on a connection (then the app
can just read as many of them as wanted at a given address).
Not a lot left to simplify.


Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/

On 10/16/2015 1:14 AM, Paul Rubin wrote:
> rickman <gnuarm@gmail.com> writes:
>> Why do I need anything special to look for errors from a USB device?
>
> Do I understand what you're trying to do here?  You want to control some
> kind of embedded gadget with a web browser--this is fine.  The gadget
> has an ethernet port so you want it to to expose web server directly to
> your LAN: good idea in principle.  It means any computer on your LAN can
> browse the gadget without any client app.  Very convenient.  The
> embedded thing has limited memory so you don't want to put in a full
> tcp/ip stack.  In particular you it to support UDP but not TCP.  This is
> strained to begin with, but it also defeats the purpose of getting rid
> of the client app, since browsers speak TCP.  And the embedded web
> application and httpd will consume some resources even without TCP.
>
> So I think the UDP scheme doesn't help you that much.  Easiest may be to
> put a dead simple text interface in the gadget (you like Forth so a
> Forth interactive session would work nicely), and have a host-side app
> (assuming you have something like an rPi talking to the gadget) that
> interacts with the gadget and presents an HTTP server and web
> application to the LAN.
>
> Is this just about the limitations of that package I linked earlier,
> that appeared to have UDP but not TCP?  There has to be TCP support for
> it, or in some alternative package.  Really, they would not have put an
> ethernet port on that board if they thought running TCP/IP on it was
> impractical.  It is fine.

Who said anything about Ethernet?  Well, maybe I did much earlier in the 
thread, but I guess my topic has wandered off from that, or maybe I 
confused this with another conversation... sorry.

Bottom line is to see what the chip vendor has provided.  They have a 
few dozen app notes on this chip and all the various features and 
support software.  It will take me a while to wade through it all.  The 
updated package for the CPU side of things won't be out until next week. 
  I'm a little worried that it is not going to be smooth sailing.

-- 

Rick

Dimiter_Popoff <dp@tgi-sci.com> wrote:

(snip)

> Multicast can be a big deal of course. And, like I said, there can be
> compelling reasons to not use tcp. VOIP and video streaming can well be
> among them, tcp will only be in the way for skipping over lost link
> periods.
> But for say file transfers or sort of telnet interactions you can
> only buy yourself more work to do by ditching tcp.
> IOW they both have their place - and exist for a reason.

I don't know the whole history, but tftp was used for many years
to boot diskless machines where the simplest code had to fit in ROM.

tftp is UDP based, not so fast as it uses small packets and a not
so efficient handshaking, but works well for file transfers when
you can't fit TCP. 

Diskless suns transfer the first level boot program with tftp,
which is enough to NFS mount and transfer a much bigger boot program.

But as far as I know, no UDP based http.

-- glen

On 16/10/15 03:19, Les Cargill wrote:
> Dimiter_Popoff wrote:
>> On 15.10.2015 г. 07:13, Roberto Waltman wrote:
>>> Dimiter_Popoff wrote:
>>>
>>>> generally you cannot serve webpages using udp. It gives you just a
>>>> "best effort" delivery, even on a local ethernet packets do get lost.
>>>
>>> Rick, Dimiter, check this:
>>>
>>> ENet - Reliable UDP networking library
>>>
>>> ENet's purpose is to provide a relatively thin, simple and robust
>>> network communication layer on top of UDP (User Datagram Protocol).
>>>
>>> The primary feature it provides is optional reliable, in-order
>>> delivery of packets.
>>>
>>> http://enet.bespin.org/
>>>
>>> R.W.
>>>
>>
>> Hi Roberto,
>>
>> I gave it a brief glance, looks interesting and is probably useful for
>> certain tasks but it is far from a replacement for tcp. From what I saw
>> in the overview I believe they ensure in order delivery by waiting for
>> the ack before sending a new packet; meaning an RTT between each two
>> packets, which may mean seconds.
>> TCP is very well thought through,it can hardly be simplified any further
>> and stay useful. For data streaming it is just "the" choice by a great
>> margin. It will take a while until all the tiny MCUs are capable
>> of it but they are closing in on that. For example my tcp/ip
>> implementation for DPS can easily fit in half a megabyte of code,
>> that including the entire OS basics (file system, window support, all
>> the system calls etc., all that uncompromised and not really planned
>> as "small footprint").
>>
>> Dimiter
>>
>>
>> ------------------------------------------------------
>> Dimiter Popoff, TGI             http://www.tgi-sci.com
>> ------------------------------------------------------
>> http://www.flickr.com/photos/didi_tgi/
>>
> 
> Tactically, UDP is preferable for real-time-ish things.
> This is especially true if the things are local to each other -
> say, no the same basically layer 2 network without much routing
> in place.

There must be an emphasis here on the "real-time-ish" rather than "real
time".  With UDP, you still have layers of ARP, ICMP, and other details
- although it is possible to reduce them greatly with static ARP tables.
 To get closer to real-time, you need to make sure that you are working
on a closed network - no Windows machines chattering away with endless
pointless broadcast telegrams.  And for the lowest overhead and most
predictable networking, use Ethernet telegrams directly rather than IP.

But as you say, UDP gives you more real-time-ish when you have a simple
local network.  In particular, you have full control of things like
timeouts, packet ordering, packet fragmentation, acknowledgements and
retries.  That means more effort in the application code to keep it all
working well, but more flexibility.

If you have more complex routing then the mostly predictable nature of
UDP is lost anyway, so TCP is usually a better choice since it works
smoother with routing setups.

> 
> You don't improve the channel entropy by using TCP; all
> you do is buy "insurance". When things go splat, they really
> go splat. You still end up needing timers and making
> guesses about state.

True, but you have fewer ways for things to go wrong with TCP (as far as
the application layer is concerned).  Either everything works perfectly,
or the connection fails completely.  You don't have to deal with all the
small details - packets getting damaged, coming in in different orders, etc.

> 
> TCP is quite efficient; there just can be long timeouts
> when some things happen and there's not much configuration
> wise you can do about it. It may be as long a five
> minutes before you are notified that a channel has actually gone
> dark in certain cases.
> 
> SNMP was implemented on top of UDP because it was
> judged that SNMP would be most important when things
> were going splat. Those guys knew what they were doing.
> 

I have never been particularly impressed with SNMP, but I haven't had
much need of it.  The biggest issue with it, in my humble and poorly
informed opinion, is that the designers failed at the very first letter
and that failure permeates the whole protocol.  It is called Simple
Network Management Protocol - but it is very far from being "simple".

(However, I agree that UDP is the appropriate choice here, rather than
TCP.  They got that bit right.)

On 16/10/15 07:14, Paul Rubin wrote:
> rickman <gnuarm@gmail.com> writes:
>> Why do I need anything special to look for errors from a USB device?
> 
> Do I understand what you're trying to do here?  You want to control some
> kind of embedded gadget with a web browser--this is fine.  The gadget
> has an ethernet port so you want it to to expose web server directly to
> your LAN: good idea in principle.  It means any computer on your LAN can
> browse the gadget without any client app.  Very convenient.  The
> embedded thing has limited memory so you don't want to put in a full
> tcp/ip stack.  In particular you it to support UDP but not TCP.  This is
> strained to begin with, but it also defeats the purpose of getting rid
> of the client app, since browsers speak TCP.  And the embedded web
> application and httpd will consume some resources even without TCP.
> 

As far as I can tell, the only reason UDP has come up is because some
people have been giving Rick a very skewed impression about the
processor and resource requirements for TCP/IP.  While it is clearly
true that this microcontroller/FPGA/SoC is not suitable for running
Linux, and it is also clearly true that UDP requires fewer resources
than TCP/IP, it is completely incorrect to suggest that this SoC will
have trouble with TCP/IP.  The SoC will handle TCP/IP without problems
(using a good, flexible and powerful stack like lwip - not just a
minimalist system like uip).

> So I think the UDP scheme doesn't help you that much.  Easiest may be to
> put a dead simple text interface in the gadget (you like Forth so a
> Forth interactive session would work nicely), and have a host-side app
> (assuming you have something like an rPi talking to the gadget) that
> interacts with the gadget and presents an HTTP server and web
> application to the LAN.
> 
> Is this just about the limitations of that package I linked earlier,
> that appeared to have UDP but not TCP?  There has to be TCP support for
> it, or in some alternative package.  Really, they would not have put an
> ethernet port on that board if they thought running TCP/IP on it was
> impractical.  It is fine.
> 

David Brown <david.brown@hesbynett.no> writes:
> I have never been particularly impressed with SNMP, but I haven't had
> much need of it.  ...  (However, I agree that UDP is the appropriate
> choice here, rather than TCP.  They got that bit right.)

I didn't realize SNMP was used much any more, except in weird old
legacy-descended devices.  I thought newer stuff tends to have embedded
web servers instead of SNMP.

On 16/10/15 09:16, Paul Rubin wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> I have never been particularly impressed with SNMP, but I haven't had
>> much need of it.  ...  (However, I agree that UDP is the appropriate
>> choice here, rather than TCP.  They got that bit right.)
> 
> I didn't realize SNMP was used much any more, except in weird old
> legacy-descended devices.  I thought newer stuff tends to have embedded
> web servers instead of SNMP.
> 

SNMP is popular for network equipment (such as managed switches).  I
haven't used it myself, more than for very simple tests long ago, but it
can be used with a wide variety of monitoring software.  The key point
here is that the same master software on the PC (or server) can use SNMP
to monitor a wide range of devices - embedded web servers are suitable
only for looking at an individual device at a time.

I am sure SNMP is a good protocol for that purpose - but naming it
"Simple" is highly misleading.  Wikipedia lists 29 RFC's specifically
for SNMP.

David Brown wrote:
> On 16/10/15 03:19, Les Cargill wrote:
>> Dimiter_Popoff wrote:
>>> On 15.10.2015 г. 07:13, Roberto Waltman wrote:
>>>> Dimiter_Popoff wrote:
>>>>
>>>>> generally you cannot serve webpages using udp. It gives you just a
>>>>> "best effort" delivery, even on a local ethernet packets do get lost.
>>>>
>>>> Rick, Dimiter, check this:
>>>>
>>>> ENet - Reliable UDP networking library
>>>>
>>>> ENet's purpose is to provide a relatively thin, simple and robust
>>>> network communication layer on top of UDP (User Datagram Protocol).
>>>>
>>>> The primary feature it provides is optional reliable, in-order
>>>> delivery of packets.
>>>>
>>>> http://enet.bespin.org/
>>>>
>>>> R.W.
>>>>
>>>
>>> Hi Roberto,
>>>
>>> I gave it a brief glance, looks interesting and is probably useful for
>>> certain tasks but it is far from a replacement for tcp. From what I saw
>>> in the overview I believe they ensure in order delivery by waiting for
>>> the ack before sending a new packet; meaning an RTT between each two
>>> packets, which may mean seconds.
>>> TCP is very well thought through,it can hardly be simplified any further
>>> and stay useful. For data streaming it is just "the" choice by a great
>>> margin. It will take a while until all the tiny MCUs are capable
>>> of it but they are closing in on that. For example my tcp/ip
>>> implementation for DPS can easily fit in half a megabyte of code,
>>> that including the entire OS basics (file system, window support, all
>>> the system calls etc., all that uncompromised and not really planned
>>> as "small footprint").
>>>
>>> Dimiter
>>>
>>>
>>> ------------------------------------------------------
>>> Dimiter Popoff, TGI             http://www.tgi-sci.com
>>> ------------------------------------------------------
>>> http://www.flickr.com/photos/didi_tgi/
>>>
>>
>> Tactically, UDP is preferable for real-time-ish things.
>> This is especially true if the things are local to each other -
>> say, no the same basically layer 2 network without much routing
>> in place.
>
> There must be an emphasis here on the "real-time-ish" rather than "real
> time".  With UDP, you still have layers of ARP, ICMP, and other details
> - although it is possible to reduce them greatly with static ARP tables.


Once the sockets are established, all that overhead is behind you.

>   To get closer to real-time, you need to make sure that you are working
> on a closed network - no Windows machines chattering away with endless
> pointless broadcast telegrams.  And for the lowest overhead and most
> predictable networking, use Ethernet telegrams directly rather than IP.
>

That's fine if you can manage it. It's extremely nonportable and
somewhat painful. Undfferenitiated sockets are generally better.

> But as you say, UDP gives you more real-time-ish when you have a simple
> local network.  In particular, you have full control of things like
> timeouts, packet ordering, packet fragmentation, acknowledgements and
> retries.  That means more effort in the application code to keep it all
> working well, but more flexibility.
>
> If you have more complex routing then the mostly predictable nature of
> UDP is lost anyway, so TCP is usually a better choice since it works
> smoother with routing setups.
>

Right.

>>
>> You don't improve the channel entropy by using TCP; all
>> you do is buy "insurance". When things go splat, they really
>> go splat. You still end up needing timers and making
>> guesses about state.
>
> True, but you have fewer ways for things to go wrong with TCP (as far as
> the application layer is concerned).  Either everything works perfectly,
> or the connection fails completely.  You don't have to deal with all the
> small details - packets getting damaged, coming in in different orders, etc.
>

TCP affords something like rational ignorance on the part of
the application.

>>
>> TCP is quite efficient; there just can be long timeouts
>> when some things happen and there's not much configuration
>> wise you can do about it. It may be as long a five
>> minutes before you are notified that a channel has actually gone
>> dark in certain cases.
>>
>> SNMP was implemented on top of UDP because it was
>> judged that SNMP would be most important when things
>> were going splat. Those guys knew what they were doing.
>>
>
> I have never been particularly impressed with SNMP, but I haven't had
> much need of it.  The biggest issue with it, in my humble and poorly
> informed opinion, is that the designers failed at the very first letter
> and that failure permeates the whole protocol.  It is called Simple
> Network Management Protocol - but it is very far from being "simple".
>

Marshall Rose would say "The protocol is simple, not
the process required to use it."

> (However, I agree that UDP is the appropriate choice here, rather than
> TCP.  They got that bit right.)
>

As they said in the Music Man, ya gotta know the territory.

-- 
Les Cargill