UDP Terminal Protocol

Robert Wessel wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
<snip>
>
> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.
>
>

If you are using UDP on a local segment, you're reasonably safe assuming 
things are in order and not fragmented unless you try to
push the envelope with large PDUs.

Over a router of unknown configuration? Not so much.

>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>> If you want something reliable that works over WANs, use TCP: it's the
>> only way you're going to dependably make anything work through a
>> firewall.
>
>
> Sure, no disagreement there.  And no, operation over WANs is not a
> prime consideration, although it certainly wouldn't hurt if it could,
> even if the user had to deal with firewall issues, or set up a relay
> server of some sort.  But the scope would be within a data-center, or
> perhaps campus (assuming at least real IP address assignment - without
> that, local LAN).  Remember the idea here is to replace a serial
> console port.
>

Serial ports don't change the order of arrival or fragment :)

>
>> If you're going to assume local Ethernet, you can just use UDP.


-- 
Les Cargill

Don Y wrote:
> On 12/14/2016 12:28 PM, Robert Wessel wrote:
<snip>
> TFTP without the "windowsize" option handles the problem just by requesting
> retransmits until you've "got" the next packet IN SEQUENCE.  Support
> for the windowsize forces the client to keep track of *which* packets
> it's received -- and tolerate the possibility that it might accumulate
> them out-of-order in how it frames its acknowledgements.
>

I am pretty sure bootp only works on local segments. Not 100% sure about 
TFTP, but I would be surprised if it survived fragmentation or
changes in sequencing.

<snip>

-- 
Les Cargill

On 12/15/2016 8:52 PM, Les Cargill wrote:
> Don Y wrote:
>> On 12/14/2016 12:28 PM, Robert Wessel wrote:
> <snip>
>> TFTP without the "windowsize" option handles the problem just by requesting
>> retransmits until you've "got" the next packet IN SEQUENCE.  Support
>> for the windowsize forces the client to keep track of *which* packets
>> it's received -- and tolerate the possibility that it might accumulate
>> them out-of-order in how it frames its acknowledgements.
>
> I am pretty sure bootp only works on local segments. Not 100% sure about TFTP,
> but I would be surprised if it survived fragmentation or
> changes in sequencing.

Classic TFTP only allows one outstanding packet.  Later revisions
allow a window size to be negotiated -- essentially allowing
multiple packets to be on the wire (from the standpoint of the
client) simultaneously.  Client-side code decides which packets
to acknowledge and when.  So, if a packet 5 (of 1-8 in this window)
arrives later than the client "planned", 1-4 are acknowledged
causing 5-12 to be sent in the next window... even if the "first" 5
arrives just after the ack has been sent.

You can workaround fragmentation by negotiating a suitable packet
size (yet another extension to the original TFT protocol).

I.e., you can conceivably configure TFTP to bridge routers
*if* that's a deliberate goal.

BOOTP only addresses the idea of centralizing the devices'
configurations (instead of having to push them into the
device at "install time").  DHCP can achieve similar results
(in a more modern approach).

[Of course, you can also bastardize any number of protocols
to come up with your own approach -- "downloading" configuration
"files" that are named (by convention) with the MAC address
of the device in question, etc.  But, you want to be cognizant of
typical "best practices" so you don't pose a system configuration
headache for the network administrator(s)]

In article <qs535ctbk361kav29rjd236dat22mo199e@4ax.com>,
Robert Wessel  <robertwessel2@yahoo.com> wrote:
>On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
><invalid@invalid.invalid> wrote:
>
>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>>I know of people who have implemented that on top of IP rather than
>>UDP.  They call it TCP. ;)
>
>
>Well, yes, but the point is that TCP is a fair chunk of code, and for
>this function it has to go in a spot where it's less than convenient
>to update.  Hence the desire for maximum simplicity.  I used TFTP as
>an example, since it solves a similar problem (and you could
>certatinly trivially implemented a TCP version of TFTP), with similar
>constraints.

A rudimentary "simple-tcp" that is not quite compliant with
TCP but will interoperate with one that is is not much code.
You will have to handle a connection or maybe a handful, but not
very many. You will have to do retransmissions, but you can back
off to a constant, semi-high value (like 6 seconds) on the second
attempt. You can deny all options, and stay with a small window
of a few k, and send 512 bytes or less.

That should reduce the requirements to a few hundred lines of
code beyond implementing UDP.

You can also have a very rudimentary telnet negotiation, basically
enforcing a standard set of options and accepting nothing else.

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.
>>
>>Just use a raw TCP connection. There's no need to implement the TELNET
>>protocol at all.  The Unix command line client will skip all the
>>handshaking stuff if you use a non-standard port.  Or you can use
>>netcat.  Or any number of terminal emulators or COM port redirectors
>>that support raw TCP connections.
>
>
>Given how little TELNET actually does (or at least, needs to do), if
>you have TCP, there would be no really issue implementing that.  Raw
>TCP is obviously an option, but again, the incremental cost to do
>actual TELNET is low.  And as I said, if you have TCP, why not just do
>this with HTTP, and really make things convenient for people.
>
>
>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>using both those approaches for close to 20 years, and have never had
>>any problems.
>
>
>Yet what exactly is the protocol?  How many characters can you put in
>a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>on or off?  With UDP (and a bit of consideration at the protocol
>layer), you can even avoid any IP configuration requirements (at least
>on a single LAN) by doing everything with broadcasts.

You probably have to have some default IP address, which you can
change. Making a simple udp one-shot protocol to set such options
should be easy.

>I've simply never implemented anything with UDP that didn't take error
>handling into account - is the common practice just to assume that you
>will not lose/reorder/duplicate/mangle any packing so long as you're
>on a local segment?  Again, TFTP addresses those issues, at least as
>they apply to its requirements.
>
>
>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>>If you want something reliable that works over WANs, use TCP: it's the
>>only way you're going to dependably make anything work through a
>>firewall.
>
>
>Sure, no disagreement there.  And no, operation over WANs is not a
>prime consideration, although it certainly wouldn't hurt if it could,
>even if the user had to deal with firewall issues, or set up a relay
>server of some sort.  But the scope would be within a data-center, or
>perhaps campus (assuming at least real IP address assignment - without
>that, local LAN).  Remember the idea here is to replace a serial
>console port.

A "simple-TCP" will handle this, and you could make a udp simple
config-setter like the ones digium does for the iaxys should be
an afternoon's work.

>>If you're going to assume local Ethernet, you can just use UDP.

You will be amazed on how much chatter, duplications and other
stuff I have found on ethernets.

-- mrr

On 2016-12-24, Morten Reistad <first@last.name.invalid> wrote:

> You can also have a very rudimentary telnet negotiation, basically
> enforcing a standard set of options and accepting nothing else.

That turns out to be surprisingly difficult.  It's not a lot of code,
but no two clients behave the same, and the first time you trie a new
client, you find a new problem.

>>Yet what exactly is the protocol?  How many characters can you put in
>>a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>>on or off?  With UDP (and a bit of consideration at the protocol
>>layer), you can even avoid any IP configuration requirements (at least
>>on a single LAN) by doing everything with broadcasts.
>
> You probably have to have some default IP address, which you can
> change. Making a simple udp one-shot protocol to set such options
> should be easy.

One clever method for establishing an initial IP address that I've
seen is that the embedded device watches for unicast IP packets sent
to its MAC address.  When it sees one, it temporarily adopts whatever
destination IP address was in the packet. That way, to initially
configure the device's IP, all you have to do is add an entry to your
host's ARP cache with the desired IP address and then ping that IP
address. IIRC, the device in question only did this if the device was
at factory defaults.

-- 
Grant

Grant Edwards <invalid@invalid.invalid> wrote:
> One clever method for establishing an initial IP address that I've
> seen is that the embedded device watches for unicast IP packets sent
> to its MAC address.  When it sees one, it temporarily adopts whatever
> destination IP address was in the packet. That way, to initially
> configure the device's IP, all you have to do is add an entry to your
> host's ARP cache with the desired IP address and then ping that IP
> address. IIRC, the device in question only did this if the device was
> at factory defaults.

Or you could just use IPv6: router broadcasts Router Advertisements
containing the local network prefix.  Client picks that up, appends its MAC
address, it now has its IP.  Everyone else can work out the IP too.
This is not special 'factory defaults' magic, it's how IPv6 works all the
time.

Theo

On 12/25/2016 6:45 AM, Theo Markettos wrote:
> Grant Edwards <invalid@invalid.invalid> wrote:
>> One clever method for establishing an initial IP address that I've
>> seen is that the embedded device watches for unicast IP packets sent
>> to its MAC address.  When it sees one, it temporarily adopts whatever
>> destination IP address was in the packet. That way, to initially
>> configure the device's IP, all you have to do is add an entry to your
>> host's ARP cache with the desired IP address and then ping that IP
>> address. IIRC, the device in question only did this if the device was
>> at factory defaults.
>
> Or you could just use IPv6: router broadcasts Router Advertisements
> containing the local network prefix.  Client picks that up, appends its MAC
> address, it now has its IP.  Everyone else can work out the IP too.
> This is not special 'factory defaults' magic, it's how IPv6 works all the
> time.

[N.B. IPv6 is a costlier stack]

The problem with all discovery/configuration services is they
can be spoofed by a malicious host ("server") introduced on the
local network.  And, unless you have an IDS in place and/or
vigilant staff monitoring ALL log files for "unusual" but
TRANSIENT (!) error messages ("address already in use", etc.),
you're unlikely to know a service has been hijacked to compromise
one or more hosts (and wonder the use for which THOSE hosts are
now targeting).

On Thu, 15 Dec 2016 11:06:19 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>My read of the OP's comments was that he wanted a means of talking with
>the "command/control console" of his device, remotely, using network
>fabric instead of a dedicated "serial" link.
>
>E.g., in a data center environment, this would be used for LOM -- in
>lieu of a "display and keyboard".


While it's moot at this point, I need to support more low-level stuff
than an operational console.  Think BIOS setup screens.

On Thu, 15 Dec 2016 21:42:42 -0600, Les Cargill
<lcargill99@comcast.com> wrote:

>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>>
>
>A bit??? I think you left a "quite" out. :) If the target
>is Linux, then it's less trouble, but it's still somewhat involved.


A basic HTTP server needs surprisingly little.  The one in our library
is a bit over 2KLOCs (and there's some utility stuff which can expand
that), but can be built in iterative, select/state-machine and
threaded models, and has a few features not strictly needed.  A
simple, iterative only, model could probably be squeezed into half
that.

Obviously it's no threat to Apache's dominance, and the web
programming is pretty low level.  I cobbled the following up in a
couple of minutes:


int p_index(void)
{
   int i;
   time_t rawtime;
   struct tm * timev;

   SendHTMLTextHeader();
   if (MethodHEAD())
       return 1;

   SendHTMLText(
       "<html><body>"
       "<p align=\"center\"><img src=\"c-logo.gif\" width=\"328\"
height=\"76\"> </p>"
       "<h1>Hello world!</h1>"
               );

   time(&rawtime);
   timev = localtime(&rawtime);
   SendHTMLprintf("It is: %s</br>", asctime(timev));

   SendHTMLprintf("The passed (%i) parameters are:</br><blockquote>",
parmcount);
   for (i=0; i<parmcount; i++)
       SendHTMLprintf("%s=%s</br>", parms[i].name, parms[i].value);
   SendHTMLText("</blockquote>");

   SendHTMLText("</body></html>");
   SendEnd();
   return 1;
}


#include "c-logo.h"
int p_c_logo_gif(void)
{
   return SendHTMLBinary(c_logo_gif_data, c_logo_gif_len,
"image/gif");
}


struct pages_t pages[] = 
  {
   {"/", p_index},
   {"/index.htm", p_index},
   {"/index.html", p_index},
   {"/c-logo.gif", p_c_logo_gif},
   {NULL, NULL}
  };


int main()
{
   int r;

   r = WebServer(pages, 2505);
   printf("rc=%d, msg=%s\n", r, GetMsg());
}


(The above would be correct for the iterative version.)

The file c-logo.h contains a GIF, a binary file converted to a .h with
a trivial utility.  Something like:

// bin2h file conversion of c-logo.gif
unsigned char c_logo_gif_data[] = {
 /* 0x0000 */  0x47,0x49,0x46,0x38,0x37,0x61,0x48,0x01,
 /* 0x0008 */  0x4c,0x00,0xf7,0x00,0x00,0xff,0xff,0xff,
 /* 0x0010 */  0xe7,0xe7,0xe7,0xde,0xde,0xde,0xd6,0xd6,
...
 /* 0x1700 */  0x2f,0x11,0x01,0x01,0x00,0x3b,
    };
long c_logo_gif_len = 5894;

On Mon, 26 Dec 2016 16:34:08 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:

>On Thu, 15 Dec 2016 11:06:19 -0700, Don Y
><blockedofcourse@foo.invalid> wrote:
>
>>My read of the OP's comments was that he wanted a means of talking with
>>the "command/control console" of his device, remotely, using network
>>fabric instead of a dedicated "serial" link.
>>
>>E.g., in a data center environment, this would be used for LOM -- in
>>lieu of a "display and keyboard".
>
>
>While it's moot at this point, I need to support more low-level stuff
>than an operational console.  Think BIOS setup screens.

This is a completely new requirement.

No way you could handle this with UDP or even TCP..

You will need some remote KVM system (such as Black Box) which
transfer the keyboard, mouse signals as well as the (VGA) scan raster
to a remote site.

If the data center device can be controlled solely over a serial link,
just install an ethernet to serial converter close to the device to be
controlled and connect to that converter using standard Telnet.