UDP Terminal Protocol

On Thu, 15 Dec 2016 01:49:38 +0000 (UTC), antispam@math.uni.wroc.pl
wrote:

>Robert Wessel <robertwessel2@yahoo.com> wrote:
>> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
>> <invalid@invalid.invalid> wrote:
>> 
>> >On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>> >
>> >> OTOH, it's clear that a number of people have implemented a UDP-based
>> >> protocol for handling the sort of things often done with a serial
>> >> console attached to a device.  This has a number of advantages - you
>> >> can fake UPD over Ethernet with very little code, especially if you
>> >> define away things like fragmentation and options.  A simplified
>> >> TELNET-ish protocol on top of that would be pretty simple, you just
>> >> need some basic packet sequencing and acknowledgements.
>> >
>> >I know of people who have implemented that on top of IP rather than
>> >UDP.  They call it TCP. ;)
>> 
>> 
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.  Hence the desire for maximum simplicity.  I used TFTP as
>> an example, since it solves a similar problem (and you could
>> certatinly trivially implemented a TCP version of TFTP), with similar
>> constraints.
>
>My experience with TFTP in early nineties was that on our
>network attempting to send file larger than 1MB will usually
>stall.  We had several segments of thin Etehrent connected
>via a repeater and used cheap equipement.  TCP run with
>quite resonable performance, but TFTP was essentially
>unusable.  Apparently TFTP specification was extended
>to get better reliability (or maybe our implemetation
>dropped optional parts), but it seems that using raw UDP
>will give you _more_ reliability than I got from TFTP.
>More preciesely, IIRC both at Ehernet level and IP level
>you get a checksum for each packet.  You are talkong
>about low performance, so presumably no need to throttle
>transmission.  At low speed in local network there
>should be no problem with reordering packets.  As my
>expample shows in practice TFTP was unable to retransmit
>missing packets.  Together this means that acknowledgements
>buy you nothing => you are end up at raw UDP level.
>
>> 
>> >> IOW, something with a relationship to TELNET sort of like TFTPs
>> >> relationship to FTP.
>> >
>> >Just use a raw TCP connection. There's no need to implement the TELNET
>> >protocol at all.  The Unix command line client will skip all the
>> >handshaking stuff if you use a non-standard port.  Or you can use
>> >netcat.  Or any number of terminal emulators or COM port redirectors
>> >that support raw TCP connections.
>> 
>> 
>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>> this with HTTP, and really make things convenient for people.
>> 
>> 
>> >If you're on a local Ethernet segment, just use raw UDP.  I've been
>> >using both those approaches for close to 20 years, and have never had
>> >any problems.
>> 
>> 
>> Yet what exactly is the protocol?  How many characters can you put in
>> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>> on or off?  With UDP (and a bit of consideration at the protocol
>> layer), you can even avoid any IP configuration requirements (at least
>> on a single LAN) by doing everything with broadcasts.
>> 
>> I've simply never implemented anything with UDP that didn't take error
>> handling into account - is the common practice just to assume that you
>> will not lose/reorder/duplicate/mangle any packing so long as you're
>> on a local segment?  Again, TFTP addresses those issues, at least as
>> they apply to its requirements.
>
>Well, maybe modern TFTP is better, but IME TFTP did _not_ address
>the isssues.


Certainly bad TFTP implementations exist, but there's really nothing
in the protocol that should permit behavior as bad as you describe.


>I think you look at your problem from wrong point of view.  Once
>you want decent capability for transmitting streams of data you
>end up with something very similar to TCP.  So better question
>is: "what parts of typical TCP implementation can be safely
>skipped and which existing implementation has good support
>for doing so".  Partial answer is: TCP implementation can be
>simplified quite a lot and still work reasonably well.
>Commer and Stevens in their 1994 book described "complete"
>implementation of Internet protocls, their code was simple
>but they tried to cover all required parts.
>They wrote that they needed about 3000 LOC for TCP
>(and slightly larger amount for mamdatory lower level parts).
>If you limit their implementation to TCP only (+ needed lower
>level parts) it will by probably 60% bigger than UDP only (again
>UDP + needed lower level parts).  Compared to 1994 TCP
>was extended, but it seems safe to assume that even in
>modern network you should be able to use implementation
>having 6000 LOC (for TCP and lower layers, but exluding
>hardware handling).  This 6000 LOC-s is likely to
>produce no more than 8-16 KB of machine code.  RAM
>overhead is of order few hundreds bytes per connection
>plus needed buffering (likely few kilobytes).  If
>you want reliable transmission you need buffering
>(to be able to retransmit) so extra requirements
>seem to be quite modest.  And if you have enough free
>RAM you probly also have enough program space (typical
>MCU-s seem to have much more program space than RAM).
>Of course, you may use tiny MCU with external
>Ethernet (or Wifi) chip providing buffering, but
>if you add such a chip to existing system maybe
>it is better to add a "communication processor"
>that handles all network aspects (ESP8266 will
>do this for Wifi).


All at least roughly true.  As an aside, though, the LOC count is
wrong in the 1994 edition.  The pie chart is correct, but they didn't
update the LOCs from the 1991 edition, despite adding OSPF and some
other stuff to the implementation.  So from the 1991 edition, you had
26% of the 15KLOC base in TCP (or ~4K, which is about the right number
for their implementation), not 20% of 15KLOCs as in the 1994 edition
(where the count should actually be been ~19KLOCs).  The 4KLOCs in
their SNMP and RIP implementations are omitable.


>The TCP/IP stacks that I know about seem to be larger
>than estimate I gave above.  But they have options
>to omit parts, so maybe you can trim them down.


Yes, in my experience they've all been bigger than that.


>Significant part of complexity of TCP comes from
>handling long streams.  If you can live with small
>HTTP pages than there is very small implementation
>at tuxgraphics.org.  The implementation is quite
>small but they do it via limiting HTTP page to
>a single Ethernet packet.  This implementation
>violates several requirements of TCP/IP, but
>seem to work quite well in practice.


Not really an option, one packet is just going to be too small for
HTTP.

On Wed, 14 Dec 2016 15:29:47 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 12/14/2016 12:28 PM, Robert Wessel wrote:
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.  Hence the desire for maximum simplicity.  I used TFTP as
>> an example, since it solves a similar problem (and you could
>> certatinly trivially implemented a TCP version of TFTP), with similar
>> constraints.
>
>TFTP runs using UDP datagrams.  It's simplicity lies in the fact
>that it has very few options (that need to be negotiated) and
>a window size of 1 packet:  next packet isn't sent until
>previous is ACK'd


Yes, that's why I mentioned it as an example of a protocol solving a
similar problem.


>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>> this with HTTP, and really make things convenient for people.
>
>Have you looked at an *actual* TELNETd implementation?  You're looking
>at several KLoC's.  You can pare this back by deciding to only
>advertise certain capabilities.  But, it's not a "weekend job"!


Yes, and I've actually done a couple.  And yes, if you just handle the
common TELNET commands (which you can mostly ignore, or handle
trivially), support a minimal number of options (ECHO, SGA, TERMTYPE,
and BINARY (maybe)?) and maybe accept and TERMINAL-TYPE you get from
an IAC/SB/TERMTYPE negotiation, it's pretty small.


>As you're *probably* not looking for maximum performance, minimum latency,
>etc. ("It's a serial port!"), you can probably deliberately tone down the
>implementation so it doesn't tax the fabric *nor* the clients involved
>(i.e., if you give a client plenty of time to process packet 1 before
>trying to send packet 2, there's a better chance that it won't DROP
>packet 2 and, thus, need to re-request it).


That would be the assumed protocol.


>[Historically, there have been NICs that would "not see" packets
>appearing too close to other packets on the wire -- think of it
>as "hardware bugs" necessitating software work-arounds]


Thank you for reminding me of 3c501s.  I had put that piece of crap
out of mind decades ago...  The 3c501 was special in that it had only
one I/O buffer, and it needed to be set to either transmit or receive
mode.  So if you sent a packet, you'd set it to transmit mode, push
the packet into the buffer, tell the NIC to send it, and then switch
the NIC into receive mode.  Unfortunately the switch was slow enough
that if whoever you sent the packet to was really fast to respond
(that required a better NIC than a 3c501 at the other end, of course),
it was not actually possible to get the local NIC into receive mode
fast enough!  So the "software workaround" was a delay at the other
end.

Of course the 3c501 had issues with too many consecutive receives as
well.

On Wed, 14 Dec 2016 15:09:13 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>> Physically, the idea would be to replace the serial port with an
>> Ethernet port.  It would be nice to be able to still connect without
>> any network infrastructure at all (beyond, possibly, a crossover
>> Ethernet cable).
>
>It might be interesting to consider putting serial port *and* ethernet
>on the same connector (I'd have to consider the hardware consequences
>to see if there's a "cheap" way of doing so).


It somewhat depends on the connector.  RS-232 and Ethernet signals
won't damage each other's ports, so some incorrect connections are not
harmful.  If you stuck with a standard RJ-45, and 10/100 Ethernet, you
have four unused pins, which is likely enough for the RS-232 side. You
could then use a standard twisted pair Ethernet patch cable for the
Ethernet connection, and a special cable for the serial side. While
the RS-232 pins would be connected to the far device (typically the
switch) in the Ethernet scenario, the extra pins should not get used,
since the device won't negotiate anything faster than 100Mb, so the
remote side should ignore anything it sees on the other four wires.

A problem with that is that usually the inexpensive way to get the
RJ-45 connector onto your device is to get a jack that integrates the
magnetics.  And those tend not to make the extra pins available.  So
you'd probably have to use discrete magnetics.

If you were willing to go to a less standard connector (perhaps an
RJ-50 - aka 10P10C), you'd have enough signals for handshaking on the
serial port as well, but you'd need custom cables for both connection
types.  And of course you'd need separate magnetics.

On 12/14/2016 11:34 PM, Robert Wessel wrote:
>>> Given how little TELNET actually does (or at least, needs to do), if
>>> you have TCP, there would be no really issue implementing that.  Raw
>>> TCP is obviously an option, but again, the incremental cost to do
>>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>>> this with HTTP, and really make things convenient for people.
>>
>> Have you looked at an *actual* TELNETd implementation?  You're looking
>> at several KLoC's.  You can pare this back by deciding to only
>> advertise certain capabilities.  But, it's not a "weekend job"!
>
> Yes, and I've actually done a couple.  And yes, if you just handle the
> common TELNET commands (which you can mostly ignore, or handle
> trivially), support a minimal number of options (ECHO, SGA, TERMTYPE,
> and BINARY (maybe)?) and maybe accept and TERMINAL-TYPE you get from
> an IAC/SB/TERMTYPE negotiation, it's pretty small.

Given that you're talking about a "console" in a "data center" (as
one application domain), I'd also consider support for encryption (not a
"standard" option in many telnet clients).  Else you risk someone
eavesdropping on the connection(s) and harvesting (console) passwords,
traffic, etc.

I.e., you might find ssh/sshd better (and more modern) alternatives.

Of course, this brings key management into the picture, etc.  Tug
on a loose thread and you never know where it will lead!  :-/

If "campus" is a generic term (i.e., not specifically tied to
academic institutions), then you're faced with a network that
potentially spans multiple buildings/floors/departments/etc.
This suggests internal firewalls, routers, etc.  (which makes
a UDP solution more problematic -- unless you station a "special
host" in each such network segment and let *them* talk to each
otehr).

You'll also have some sort of IT department with which to contend.
And, "local policies".  I.e., certain ports/protocols might be
blocked as a matter of course.

This suggests you may need to offer additional configuration
options for each node.  Which begs the issue of how you maintain
and distribute that configuration information.

It might lead to using something like a BOOTPd -- or DHCPd as the
more modern/flexible alternative.  If you're expecting *many* of these
at a site, then maintenance approaches that lend themselves to
applying a template to a whole set of devices (e.g., a DHCP "class")
can have considerable appeal to the bloke charged with keeping
them all running, NOT having to read MACs printed in 3pt type
off of labels affixed to the underside of devices, etc.

The point of all this is that the cost of TCP might be a small
fraction of the total "consequences" of adopting an ethernet
solution (over a more straight-forward "serial port" approach).

So, instead, you might try approaching the problem with a different
set of assumptions:  assume you *have* a full stack (with some
particular set of capabilities, yet to be quantified).  With that,
what might you do differently?  And, why?  Is it still appropriate
to think of it as providing a "serial port for interactive use"?
Or, might there be other features for which you can leverage this
capability?

>> As you're *probably* not looking for maximum performance, minimum latency,
>> etc. ("It's a serial port!"), you can probably deliberately tone down the
>> implementation so it doesn't tax the fabric *nor* the clients involved
>> (i.e., if you give a client plenty of time to process packet 1 before
>> trying to send packet 2, there's a better chance that it won't DROP
>> packet 2 and, thus, need to re-request it).
>
> That would be the assumed protocol.

That is how RLOGIN works (but, over TCP)

>> [Historically, there have been NICs that would "not see" packets
>> appearing too close to other packets on the wire -- think of it
>> as "hardware bugs" necessitating software work-arounds]
>
> Thank you for reminding me of 3c501s.  I had put that piece of crap
> out of mind decades ago...  The 3c501 was special in that it had only
> one I/O buffer, and it needed to be set to either transmit or receive
> mode.  So if you sent a packet, you'd set it to transmit mode, push
> the packet into the buffer, tell the NIC to send it, and then switch
> the NIC into receive mode.  Unfortunately the switch was slow enough
> that if whoever you sent the packet to was really fast to respond
> (that required a better NIC than a 3c501 at the other end, of course),
> it was not actually possible to get the local NIC into receive mode
> fast enough!  So the "software workaround" was a delay at the other
> end.
>
> Of course the 3c501 had issues with too many consecutive receives as
> well.

Actually, I was thinking at the component level -- specifically,
the LANCE.

But, my point was for you to fold your hardware considerations
into the design of the stack -- instead of trying to work-around them
outside its scope.  I'm guessing you're targeting a SoC implementation...?
A peek at any outstanding "issues" for the chosen target might save
you hassles, later.

On Tue, 13 Dec 2016 04:23:16 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:

>Many devices have had a serial port for attaching a console of some
>sort, for low-level configuration/diagnostic work.  While very simple
>to implement, serial ports have issues which don't need repeating.
>
>An obvious alternative is to implement the "serial console" as a
>TELNET server.  Conceptually this is trivial, not to mention well
>standardized and well understood.  OTOH, this piles on a fair bit of
>complexity, and requires something akin to a full TCP/IP stack.  And
>if you're going to do that, you might as well just go for a web-based
>admin, it's only going to be a bit more work.

If you just need a primitive command line debugger for a small device
with ethernet port but no serial port, the simplest would be to hack
the printf() in the small device with a routine that sends a raw
Ethernet package or an UDP packet. 

On the bigger client machine, you would need a routine that reads a
command line (e.g. to facilitate command line editing) and when the
user hits the CR key, a raw ethernet or UDP frame is sent and
processed by the small server. 

An even simpler case, setup Wireshark capture filter to listen for the
MAC address of the small device. In this case, the only thing that the
"telnet" needs to do is to read a command line and send it out as a
raw ethernet or UDP frame. 

If there is only one connection point, raw Ethernet with MAC addresses
should be enough, but if multiple command points are needed in the
future, including the IP header with port numbers will help, but then
you would have to implement the ARP o the small device.

On Wed, 14 Dec 2016 14:03:59 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:


>Sure: I'd like to pare-back the network stack by not implementing TCP.
>;-)  Limited UDP-only on top of Ethernet is pretty trivial.

I guess that you are aiming of some single chip microcontrollers with
internal RAM only but with on-chip Ethernet interface. 

Of course, you will need two 1500 byte buffers for the Ethernet Rx and
Tx frames,  Raw Ethernet will need only marginally more and the UDP
implementation doesn't meed much more for multiple port number
support.

On 2016-12-15, Paul Rubin <no.email@nospam.invalid> wrote:
> Grant Edwards <invalid@invalid.invalid> writes:
>> you've spent more time than you would have just doing a simple TCP
>> implementation.
>
> I still don't understand why the OP would want to "do" (i.e. write) a
> simple TCP implementation instead of downloading one.

The OP was complaining about how much work it would be to implement
TCP, so I was comparing that to what he was proposing.  Using an
open-source TCP/IP implementation makes the comparison come out even
more in favor of TCP.

> There are lots,
> e.g.:
>
> https://en.wikipedia.org/wiki/UIP_(micro_IP)

I've not used uIP, but it's by Adam Dunkels so it gets a few points
right away in favor.  I've used his protothreads library for stackless
multitasking on a uController with only a couple hundred bytes of RAM,
and it was very cool, very well done, and very useful.

-- 
Grant Edwards               grant.b.edwards        Yow! Why are these athletic
                                  at               shoe salesmen following
                              gmail.com            me??

On 12/14/2016 11:50 PM, Robert Wessel wrote:
> On Wed, 14 Dec 2016 15:09:13 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>>> Physically, the idea would be to replace the serial port with an
>>> Ethernet port.  It would be nice to be able to still connect without
>>> any network infrastructure at all (beyond, possibly, a crossover
>>> Ethernet cable).
>>
>> It might be interesting to consider putting serial port *and* ethernet
>> on the same connector (I'd have to consider the hardware consequences
>> to see if there's a "cheap" way of doing so).
>
> It somewhat depends on the connector.  RS-232 and Ethernet signals
> won't damage each other's ports, so some incorrect connections are not
> harmful.  If you stuck with a standard RJ-45, and 10/100 Ethernet, you
> have four unused pins, which is likely enough for the RS-232 side. You

I was thinking more in terms of *sharing* conductors between interfaces.
See my previous '"On-line" 100BaseTX hardware diagnostics' post as
its along the same lines -- using a second I/F to leverage the existing
*wiring* of the first in the event of a failure of the first.  I.e.,
ethernet has failed but can I still exploit the existing WIRING
to talk to other nodes -- albeit at a crippled rate.

The flip side of this is allowing either interface to appear on
a single connector, moving the "user's choice" into the
cable with which he connects to your device (one terminated in
a DB9/25, the other in an RJ45)

> could then use a standard twisted pair Ethernet patch cable for the
> Ethernet connection, and a special cable for the serial side. While
> the RS-232 pins would be connected to the far device (typically the
> switch) in the Ethernet scenario, the extra pins should not get used,
> since the device won't negotiate anything faster than 100Mb, so the
> remote side should ignore anything it sees on the other four wires.
>
> A problem with that is that usually the inexpensive way to get the
> RJ-45 connector onto your device is to get a jack that integrates the
> magnetics.  And those tend not to make the extra pins available.  So
> you'd probably have to use discrete magnetics.
>
> If you were willing to go to a less standard connector (perhaps an
> RJ-50 - aka 10P10C), you'd have enough signals for handshaking on the
> serial port as well, but you'd need custom cables for both connection
> types.  And of course you'd need separate magnetics.
>

On 12/15/2016 3:56 AM, upsidedown@downunder.com wrote:
> On Tue, 13 Dec 2016 04:23:16 -0600, Robert Wessel
> <robertwessel2@yahoo.com> wrote:
>
>> Many devices have had a serial port for attaching a console of some
>> sort, for low-level configuration/diagnostic work.  While very simple
>> to implement, serial ports have issues which don't need repeating.
>>
>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>
> If you just need a primitive command line debugger for a small device
> with ethernet port but no serial port, the simplest would be to hack
> the printf() in the small device with a routine that sends a raw
> Ethernet package or an UDP packet.

My read of the OP's comments was that he wanted a means of talking with
the "command/control console" of his device, remotely, using network
fabric instead of a dedicated "serial" link.

E.g., in a data center environment, this would be used for LOM -- in
lieu of a "display and keyboard".

Old server farms were littered with KVM's and rack-mounted "terminals"
(fold-away LCD monitor plus keyboard wired to the KVM).  But, that required
users to be present in the cold aisle to "manage" the servers.

> On the bigger client machine, you would need a routine that reads a
> command line (e.g. to facilitate command line editing) and when the
> user hits the CR key, a raw ethernet or UDP frame is sent and
> processed by the small server.
>
> An even simpler case, setup Wireshark capture filter to listen for the
> MAC address of the small device. In this case, the only thing that the
> "telnet" needs to do is to read a command line and send it out as a
> raw ethernet or UDP frame.
>
> If there is only one connection point, raw Ethernet with MAC addresses
> should be enough, but if multiple command points are needed in the
> future, including the IP header with port numbers will help, but then
> you would have to implement the ARP o the small device.
>

Robert Wessel wrote:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
>
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
>

A bit??? I think you left a "quite" out. :) If the target
is Linux, then it's less trouble, but it's still somewhat involved.

> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.

Easily done.

> A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements.

I am not sure you even really need that. SNMP doesn't need it; it has
the "as if simultaneous" rule; when multiple PDus are used to define
an essentially atomic operation, they are to be treated as if they all
got there the same time. It's not assumed that the sender sent them in
order or without some other operation concurrently.

It is worth defining use cases carefully for such a protocol to keep
half-done things and race type[1] conditions to a minimum.

[1] unless you wish to assume only one session at a time.

Then again, there's nothing wrong with sequence numbers, but it's
sort of non-Telnet-ish if you do that.

> IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?
>

There isn't one that I know of. But it is essentially quite easy.

-- 
Les Cargill