UDP Terminal Protocol

On Tue, 13 Dec 2016 12:53:57 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 12/13/2016 3:23 AM, Robert Wessel wrote:
>> Many devices have had a serial port for attaching a console of some
>> sort, for low-level configuration/diagnostic work.  While very simple
>> to implement, serial ports have issues which don't need repeating.
>>
>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>>
>> OTOH, it's clear that a number of people have implemented a UDP-based
>> protocol for handling the sort of things often done with a serial
>> console attached to a device.  This has a number of advantages - you
>> can fake UPD over Ethernet with very little code, especially if you
>> define away things like fragmentation and options.  A simplified
>> TELNET-ish protocol on top of that would be pretty simple, you just
>> need some basic packet sequencing and acknowledgements. IOW, something
>> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>>
>> OTOH, despite looking, I just don't see any standards for doing that
>> sort of thing.  Am I just missing it?  Is there a defacto standard
>> even if there's no RFC?
>
>RLOGIN is much lighter-weight than TELNET (YMMV).  But, as with TELNET,
>runs atop TCP.


Not really sure I'd agree with that, but both are pretty simple.  If I
had TCP, doing either TELNET or RLOGIN (or HTTP) is not much of an
issue.  OTOH, there doesn't really seem to be much advantage for doing
RLOGIN over TELNET.


>What criteria do you place on the operating environment?  E.g., you
>can't run a bare "serial port" 1000km.


No, but one would hope to do something with a bit more range than a
serial port (which is what's there now), and a bit more convenient
infrastructure-wise.  Design "range" should be something on the
data-center to campus scale.  One of the prime considerations is the
ability to manage many devices without the complexity of attaching to
many serial ports scattered around the data center.


>  Are you just trying to exploit
>an existing connector (RJ45) for a secondary purpose with a LOCAL
>"console"?  I.e., do you need flow control, fragmentation/packet reassembly,
>routing, etc.?  Or, will you just slap some "special device" on the
>wire "in close proximity to" the device in question?


I don't really want a special device, although a PC with special
software is reasonable, but at worst case you'd want that PC to need
to be no closer than somewhere on the local LAN (data center or
campus, as mentioned above, would be better).  If you didn't mind a
special device, there are RS-232-to-TELNET devices you could just plug
into the serial port, and define the whole problem away.  As to flow
control and whatnot, those issue should be kept to a minimum (again
TFTP is a parallel), but since this has been done before, I'm trying
to figure out if there's existing practice to piggyback on.  If, for
example, there's a semi-standard way to do this, there are likely
terminal emulators that already support that.

The big advantage to going "whole hog" and doing TCP and TELNET (or
TCP and HTTP), is that you open this up to the whole Internet, and an
untold number of existing clients for any conceivable platform.

Physically, the idea would be to replace the serial port with an
Ethernet port.  It would be nice to be able to still connect without
any network infrastructure at all (beyond, possibly, a crossover
Ethernet cable).



>Does the interface have to be interactive?  E.g., you could leverage
>something akin to TFTP to push a "screen" of data to/from a console
>device (treat the screen as a "form" that is filled in instead of
>as a STREAM).


The model under consideration is a replacement of a serial console
assumed to be a VT-100 compatible device.  I don't think a hack like
you're suggesting is viable.  At that point we're talking about custom
software on both ends, so a "proper" TELNET-ish* protocol over UDP
would be simpler.  Developing such a (new) protocol is no problem.


*And let's not get too hung up on the specifics of TELNET here - some
bits of handshake information probably need to be exchanged at the top
of the session, and then it's just a packaging of the bi-directional
serial stream.


>You can also implement a much pared-back network stack by putting
>other constraints on how you plan to USE that stack (limit number
>of open sockets, window size, etc.)  E.g., my devices are intended to
>talk to specific *other* devices so I don't have to maintain ARP
>caches, multiple endpoints, etc.


Sure: I'd like to pare-back the network stack by not implementing TCP.
;-)  Limited UDP-only on top of Ethernet is pretty trivial.

W dniu wtorek, 13 grudnia 2016 11:22:52 UTC+1 użytkownik robert...@yahoo.com napisał:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
> 
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
> 
> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.  A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements. IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
> 
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?

If you are interested in control and diagnosis of FPGA based systems, then you may take a look at IPbus: https://svnweb.cern.ch/trac/cactus
However, as this is comp.arch.embedded, not comp.arch.fpga, that probably won't help you too much.

Regards,
Wojtek

On 12/14/2016 1:03 PM, Robert Wessel wrote:
> On Tue, 13 Dec 2016 12:53:57 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>> RLOGIN is much lighter-weight than TELNET (YMMV).  But, as with TELNET,
>> runs atop TCP.
>
> Not really sure I'd agree with that, but both are pretty simple.  If I
> had TCP, doing either TELNET or RLOGIN (or HTTP) is not much of an
> issue.  OTOH, there doesn't really seem to be much advantage for doing
> RLOGIN over TELNET.

Just to make sure there's no misunderstanding, RLOGIN runs over TCP;
you'd use it in lieu of TELNET.  (a full TELNET implementation is
many thousands of lines of code -- much of which you probably won't
need if you just want to emulate a serial port)

>> What criteria do you place on the operating environment?  E.g., you
>> can't run a bare "serial port" 1000km.
>
> No, but one would hope to do something with a bit more range than a
> serial port (which is what's there now), and a bit more convenient
> infrastructure-wise.  Design "range" should be something on the
> data-center to campus scale.  One of the prime considerations is the
> ability to manage many devices without the complexity of attaching to
> many serial ports scattered around the data center.

Once you "leave the room", things typically get complicated (unless
you're willing to live with a 100m cable limitation).  And, once you
start trying to talk to existing fabric, you have even less leeway
in terms of the corners you can cut (e.g., what happens if some
host tries to ping your node?  open an FTP connection?  etc -- even
if wrong in attempting to do so).

Many years ago (10+) I put together a proposal for a low cost "console server"
(essentially a one-port terminal server) to facilitate LOM in a datacenter
environment.  At the time, I could enhance security and cut cost by
bastardizing the network protocols with the caveat:  all of these devices will
reside on an isolated internet (so they didn't have to contend with the
variety of traffic they might encounter on a "live" internet).

Nowadays, SoC's have far more resources so even a complete stack isn't
going to break the bank (unless you're *really* scraping the bottom
of the cost/commodity barrel).

>>  Are you just trying to exploit
>> an existing connector (RJ45) for a secondary purpose with a LOCAL
>> "console"?  I.e., do you need flow control, fragmentation/packet reassembly,
>> routing, etc.?  Or, will you just slap some "special device" on the
>> wire "in close proximity to" the device in question?
>
> I don't really want a special device, although a PC with special

The "PC" was the intent of my reference  :>  The point being, it's not
an OTS solution (you still need to "develop" that end of the link).

> software is reasonable, but at worst case you'd want that PC to need
> to be no closer than somewhere on the local LAN (data center or
> campus, as mentioned above, would be better).  If you didn't mind a
> special device, there are RS-232-to-TELNET devices you could just plug
> into the serial port, and define the whole problem away.  As to flow
> control and whatnot, those issue should be kept to a minimum (again
> TFTP is a parallel), but since this has been done before, I'm trying
> to figure out if there's existing practice to piggyback on.  If, for
> example, there's a semi-standard way to do this, there are likely
> terminal emulators that already support that.

There have been lots of attempts at making "console servers" (my
reference above).  HP had a "Secure Web Console" that presented
a "terminal emulator" in a web browser window that was tethered
to the remote device's serial port.

[It was a poorly designed product, though -- particularly the
laughable "secure" aspect of it!  :> ]

You can buy single port terminal servers in a package not
much larger than a DB25 backshell (the proposal I outlined above
called for a DB9 backshell package as most servers now have 9 pin
serial ports so you don't have room for a "box" behind the server)

> The big advantage to going "whole hog" and doing TCP and TELNET (or
> TCP and HTTP), is that you open this up to the whole Internet, and an
> untold number of existing clients for any conceivable platform.

As Devil's Advocate, I'd point out that you now present a larger
attack surface and a (potentially) bigger adversary pool; one
thing with which a simple serial port didn't have to contend!

> Physically, the idea would be to replace the serial port with an
> Ethernet port.  It would be nice to be able to still connect without
> any network infrastructure at all (beyond, possibly, a crossover
> Ethernet cable).

It might be interesting to consider putting serial port *and* ethernet
on the same connector (I'd have to consider the hardware consequences
to see if there's a "cheap" way of doing so).

>> Does the interface have to be interactive?  E.g., you could leverage
>> something akin to TFTP to push a "screen" of data to/from a console
>> device (treat the screen as a "form" that is filled in instead of
>> as a STREAM).
>
> The model under consideration is a replacement of a serial console
> assumed to be a VT-100 compatible device.  I don't think a hack like
> you're suggesting is viable.  At that point we're talking about custom
> software on both ends, so a "proper" TELNET-ish* protocol over UDP
> would be simpler.  Developing such a (new) protocol is no problem.

If you were willing to make the "special device" a PC, then this
just changes the nature of the software running in the PC (given
that the "device end" is already negotiable).

I offer it only because I've frequently designed "screen oriented"
user interfaces (think: menu/dialog system atop curses(1)) so the
UI is always a ~2KW buffer in the device's memory that just gets
painted (via ANSI escape sequences) onto the hardware TTY.  Pushing
those 4KB out through a set of ~8 UDP packets (on demand any time
user asks for the screen to be repainted) would be a piece of cake;
much simpler than the curses stack residing in the device -- or the
ANSI emulator in the PC!

[The Sun Ray operated in a similar fashion; and, AFAICT, the Pano
cubes (no CPU -- entirely FPGA based) use a similar approach to
"painting a screen"]

> *And let's not get too hung up on the specifics of TELNET here - some
> bits of handshake information probably need to be exchanged at the top
> of the session, and then it's just a packaging of the bi-directional
> serial stream.

If you want to interoperate with other devices, you may have to implement
bits of the protocol that you'd hoped you could elide.  E.g., MS's
TELNET client is a PITA for me to use, here -- so I install third-party
TELNET clients that "work".

>> You can also implement a much pared-back network stack by putting
>> other constraints on how you plan to USE that stack (limit number
>> of open sockets, window size, etc.)  E.g., my devices are intended to
>> talk to specific *other* devices so I don't have to maintain ARP
>> caches, multiple endpoints, etc.
>
> Sure: I'd like to pare-back the network stack by not implementing TCP.
> ;-)  Limited UDP-only on top of Ethernet is pretty trivial.

But, even with TCP, there are economies available.  If you try to
cheap out and port an existing stack (FOSS), you will typically end
up with more resources (and features) than you might otherwise need.

IMO, the biggest downside is you'll typically insulate yourself from
the inner workings of the stack and, thus, not be aware of the sorts of
exploits that lie in wait.  Or, the costs (space/performance) that you
are bearing.

Thereafter, you have to track new releases of that codebase to see
if/when to fold them into one of your "updates".  This could result in
your taking on additional functionality/resource requirements that
you might not want.

OTOH, if you "take ownership" of such a codebase (i.e., start tweeking
it to *fit* your needs), then you're still stuck watching the releases
and now have to merge "pertinent" changes into your codebase.

[I.e., you quickly start to feel like the dog's *tail*!]

If you haven't, already, Comer's XINU (vol2) is a lightweight introduction
to the stacks; followed by his _Internetworking with TCP/IP_ series.
With those behind you, Stevens' _TCP/IP Illustrated_ is by far the best
reference.

[Though Stevens' texts tend to describe more efficient (from the
standpoint of performance) implementations; Comer's tend to be much
easier to visualize -- more *discrete* layers (and, thus, less
efficient)]

And, watch the CERT advisories to see how profoundly varied attacks
on stacks can be, in practice!

On 12/14/2016 12:28 PM, Robert Wessel wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
>
>> On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>> I know of people who have implemented that on top of IP rather than
>> UDP.  They call it TCP. ;)
>
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.  Hence the desire for maximum simplicity.  I used TFTP as
> an example, since it solves a similar problem (and you could
> certatinly trivially implemented a TCP version of TFTP), with similar
> constraints.

TFTP runs using UDP datagrams.  It's simplicity lies in the fact
that it has very few options (that need to be negotiated) and
a window size of 1 packet:  next packet isn't sent until
previous is ACK'd

[lie:  newer RFC's extend TFTP to support multiple outstanding
packets; but, this means a more complicated implementation]

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.
>>
>> Just use a raw TCP connection. There's no need to implement the TELNET
>> protocol at all.  The Unix command line client will skip all the
>> handshaking stuff if you use a non-standard port.  Or you can use
>> netcat.  Or any number of terminal emulators or COM port redirectors
>> that support raw TCP connections.
>
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.  And as I said, if you have TCP, why not just do
> this with HTTP, and really make things convenient for people.

Have you looked at an *actual* TELNETd implementation?  You're looking
at several KLoC's.  You can pare this back by deciding to only
advertise certain capabilities.  But, it's not a "weekend job"!

[I assume the TELNET client is something you'd offload to the user's
responsibility]

And, with a "virtual serial port", other issues crop up that
you may not have anticipated (if you're in a "hardware serial port"
mindset).  E.g., what happens if TWO clients try to connect to your
TELNETd?  Is that a *feature*?  Have you anticipated locks on
the various structs/parameters that the user will be modifying
via these interfaces?  Have you addressed the possibility of a
TELNET session "hanging" and the held locks never being released?

Etc.

[All of this is obvious -- in hindsight!  :> ]

>> If you're on a local Ethernet segment, just use raw UDP.  I've been
>> using both those approaches for close to 20 years, and have never had
>> any problems.
>
> Yet what exactly is the protocol?  How many characters can you put in
> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
> on or off?  With UDP (and a bit of consideration at the protocol
> layer), you can even avoid any IP configuration requirements (at least
> on a single LAN) by doing everything with broadcasts.

If you're trying to emulate a serial port, you probably won't *need*
to pack many characters into a datagram.  Figure 500 bytes is "always
safe" and don't even bother trying to negotiate a larger datagram
size.

> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.

TFTP without the "windowsize" option handles the problem just by requesting
retransmits until you've "got" the next packet IN SEQUENCE.  Support
for the windowsize forces the client to keep track of *which* packets
it's received -- and tolerate the possibility that it might accumulate
them out-of-order in how it frames its acknowledgements.

"No free lunch"

As you're *probably* not looking for maximum performance, minimum latency,
etc. ("It's a serial port!"), you can probably deliberately tone down the
implementation so it doesn't tax the fabric *nor* the clients involved
(i.e., if you give a client plenty of time to process packet 1 before
trying to send packet 2, there's a better chance that it won't DROP
packet 2 and, thus, need to re-request it).

[Historically, there have been NICs that would "not see" packets
appearing too close to other packets on the wire -- think of it
as "hardware bugs" necessitating software work-arounds]

Robert Wessel <robertwessel2@yahoo.com> writes:
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.

I still don't see the problem: TCP has been around for decades and there
are some tiny implementations, especially if you don't need multiple
packets in the air at the same time etc.  Why not just use one?

Robert Wessel <robertwessel2@yahoo.com> wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
> 
> >On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
> >
> >> OTOH, it's clear that a number of people have implemented a UDP-based
> >> protocol for handling the sort of things often done with a serial
> >> console attached to a device.  This has a number of advantages - you
> >> can fake UPD over Ethernet with very little code, especially if you
> >> define away things like fragmentation and options.  A simplified
> >> TELNET-ish protocol on top of that would be pretty simple, you just
> >> need some basic packet sequencing and acknowledgements.
> >
> >I know of people who have implemented that on top of IP rather than
> >UDP.  They call it TCP. ;)
> 
> 
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.  Hence the desire for maximum simplicity.  I used TFTP as
> an example, since it solves a similar problem (and you could
> certatinly trivially implemented a TCP version of TFTP), with similar
> constraints.

My experience with TFTP in early nineties was that on our
network attempting to send file larger than 1MB will usually
stall.  We had several segments of thin Etehrent connected
via a repeater and used cheap equipement.  TCP run with
quite resonable performance, but TFTP was essentially
unusable.  Apparently TFTP specification was extended
to get better reliability (or maybe our implemetation
dropped optional parts), but it seems that using raw UDP
will give you _more_ reliability than I got from TFTP.
More preciesely, IIRC both at Ehernet level and IP level
you get a checksum for each packet.  You are talkong
about low performance, so presumably no need to throttle
transmission.  At low speed in local network there
should be no problem with reordering packets.  As my
expample shows in practice TFTP was unable to retransmit
missing packets.  Together this means that acknowledgements
buy you nothing => you are end up at raw UDP level.

> 
> >> IOW, something with a relationship to TELNET sort of like TFTPs
> >> relationship to FTP.
> >
> >Just use a raw TCP connection. There's no need to implement the TELNET
> >protocol at all.  The Unix command line client will skip all the
> >handshaking stuff if you use a non-standard port.  Or you can use
> >netcat.  Or any number of terminal emulators or COM port redirectors
> >that support raw TCP connections.
> 
> 
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.  And as I said, if you have TCP, why not just do
> this with HTTP, and really make things convenient for people.
> 
> 
> >If you're on a local Ethernet segment, just use raw UDP.  I've been
> >using both those approaches for close to 20 years, and have never had
> >any problems.
> 
> 
> Yet what exactly is the protocol?  How many characters can you put in
> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
> on or off?  With UDP (and a bit of consideration at the protocol
> layer), you can even avoid any IP configuration requirements (at least
> on a single LAN) by doing everything with broadcasts.
> 
> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.

Well, maybe modern TFTP is better, but IME TFTP did _not_ address
the isssues.

I think you look at your problem from wrong point of view.  Once
you want decent capability for transmitting streams of data you
end up with something very similar to TCP.  So better question
is: "what parts of typical TCP implementation can be safely
skipped and which existing implementation has good support
for doing so".  Partial answer is: TCP implementation can be
simplified quite a lot and still work reasonably well.
Commer and Stevens in their 1994 book described "complete"
implementation of Internet protocls, their code was simple
but they tried to cover all required parts.
They wrote that they needed about 3000 LOC for TCP
(and slightly larger amount for mamdatory lower level parts).
If you limit their implementation to TCP only (+ needed lower
level parts) it will by probably 60% bigger than UDP only (again
UDP + needed lower level parts).  Compared to 1994 TCP
was extended, but it seems safe to assume that even in
modern network you should be able to use implementation
having 6000 LOC (for TCP and lower layers, but exluding
hardware handling).  This 6000 LOC-s is likely to
produce no more than 8-16 KB of machine code.  RAM
overhead is of order few hundreds bytes per connection
plus needed buffering (likely few kilobytes).  If
you want reliable transmission you need buffering
(to be able to retransmit) so extra requirements
seem to be quite modest.  And if you have enough free
RAM you probly also have enough program space (typical
MCU-s seem to have much more program space than RAM).
Of course, you may use tiny MCU with external
Ethernet (or Wifi) chip providing buffering, but
if you add such a chip to existing system maybe
it is better to add a "communication processor"
that handles all network aspects (ESP8266 will
do this for Wifi).

The TCP/IP stacks that I know about seem to be larger
than estimate I gave above.  But they have options
to omit parts, so maybe you can trim them down.

Significant part of complexity of TCP comes from
handling long streams.  If you can live with small
HTTP pages than there is very small implementation
at tuxgraphics.org.  The implementation is quite
small but they do it via limiting HTTP page to
a single Ethernet packet.  This implementation
violates several requirements of TCP/IP, but
seem to work quite well in practice.


-- 
                              Waldek Hebisch

On 2016-12-14, Robert Wessel <robertwessel2@yahoo.com> wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
><invalid@invalid.invalid> wrote:
>
>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>> I know of people who have implemented that on top of IP rather than
>> UDP.  They call it TCP. ;)
>
> Well, yes, but the point is that TCP is a fair chunk of code,

By the time you do "packet sequencing and acknowledgements" for both
target and host side, test it, revisit the bad protocol design
decisions, and then re-implement it for both sides you've spent more
time than you would have just doing a simple TCP implementation.

And in the end you've got something that's non-standard, non-portable,
probably still doesn't work right, and for which Wireshark doesn't
have a packet dissector.

Don't underestimate the importance of that last one.

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.

What features of TELNET do you need?

>>Just use a raw TCP connection. There's no need to implement the TELNET
>>protocol at all.  The Unix command line client will skip all the
>>handshaking stuff if you use a non-standard port.  Or you can use
>>netcat.  Or any number of terminal emulators or COM port redirectors
>>that support raw TCP connections.
>
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.

In my experinece the incremental value of Telnet is negative, since
various Telnet clients behave it vastly different and bizarre ways.

YMMV.

> And as I said, if you have TCP, why not just do this with HTTP, and
> really make things convenient for people.

OK.

>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>using both those approaches for close to 20 years, and have never had
>>any problems.
>
> Yet what exactly is the protocol?  

Um... UDP.

> How many characters can you put in a packet?

UDP says 65536 bytes per datagram.  UDP will fragment that and
reassemble it.  If you want to do a stripped-down UDP iplimentation,
limit it to about 1200.

> Any sequencing or recovery?

Nope.

> Acknowledgements?

Nope.

> Checksums on or off?

For a local Ethernte segment, you let the MAC layer worry about that.

> With UDP (and a bit of consideration at the protocol layer), you can
> even avoid any IP configuration requirements (at least on a single
> LAN) by doing everything with broadcasts.

Some IP stacks won't receive UDP broadcast packets with a
non-reachable source IP.

> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?

Unless your Ethernet segment is pretty heavily loaded, I find you can
ignore all that.

If you can't, then you need the equivalent of TCP.

-- 
Grant

Grant Edwards <invalid@invalid.invalid> writes:
> you've spent more time than you would have just doing a simple TCP
> implementation.

I still don't understand why the OP would want to "do" (i.e. write) a
simple TCP implementation instead of downloading one.  There are lots,
e.g.:

https://en.wikipedia.org/wiki/UIP_(micro_IP)

On Thu, 15 Dec 2016 02:06:18 +0000 (UTC), Grant Edwards
<invalid@invalid.invalid> wrote:

>On 2016-12-14, Robert Wessel <robertwessel2@yahoo.com> wrote:
>> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
>><invalid@invalid.invalid> wrote:
>>
>>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>>
>>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>>> protocol for handling the sort of things often done with a serial
>>>> console attached to a device.  This has a number of advantages - you
>>>> can fake UPD over Ethernet with very little code, especially if you
>>>> define away things like fragmentation and options.  A simplified
>>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>>> need some basic packet sequencing and acknowledgements.
>>>
>>> I know of people who have implemented that on top of IP rather than
>>> UDP.  They call it TCP. ;)
>>
>> Well, yes, but the point is that TCP is a fair chunk of code,
>
>By the time you do "packet sequencing and acknowledgements" for both
>target and host side, test it, revisit the bad protocol design
>decisions, and then re-implement it for both sides you've spent more
>time than you would have just doing a simple TCP implementation.


Not if this is limited, as is (basic) TFTP, to a single outstanding
packet (in each direction).  A simple sequence number is all you need,
and the ability to retransmit one packet, plus a timer for detecting
missing acks.  I've done all that before, more than once.


>And in the end you've got something that's non-standard, non-portable,
>probably still doesn't work right, and for which Wireshark doesn't
>have a packet dissector.
>
>Don't underestimate the importance of that last one.


While I've never written a decode for Wireshark, I've written several
for other packet monitors, I suspect I'd actually enjoy that little
task.  ;-)  OTOH, given the simplicity of the protocol, it's value
would be moderate.


>>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>>> relationship to FTP.
>
>What features of TELNET do you need?


None really.  The idea would be to assume a simple bidirectional
stream to a VT-100 (or heck, even a TTY in many cases).  Just enough
of TELNET to allow standard clients to be used.  If this were a custom
protocol, I'd at least have a version exchange somewhere at session
setup.


>>>Just use a raw TCP connection. There's no need to implement the TELNET
>>>protocol at all.  The Unix command line client will skip all the
>>>handshaking stuff if you use a non-standard port.  Or you can use
>>>netcat.  Or any number of terminal emulators or COM port redirectors
>>>that support raw TCP connections.
>>
>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.
>
>In my experinece the incremental value of Telnet is negative, since
>various Telnet clients behave it vastly different and bizarre ways.
>
>YMMV.


Well, most of them at least manage a basic VT-100 connection, and most
of their (TELNET) responses, odd or not, could be mostly ignored in
this implementation.  I've been reconsidering the notion of a raw-TCP
stream support, this appears to be a rather more common option in
TELNET clients than I had expected.  Still doing a minimalist TELNETD
would not be much work, and if it significantly improved
compatibility, would probably be worth it.


>> And as I said, if you have TCP, why not just do this with HTTP, and
>> really make things convenient for people.
>
>OK.
>
>>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>>using both those approaches for close to 20 years, and have never had
>>>any problems.
>>
>> Yet what exactly is the protocol?  
>
>Um... UDP.
>
>> How many characters can you put in a packet?
>
>UDP says 65536 bytes per datagram.  UDP will fragment that and
>reassemble it.  If you want to do a stripped-down UDP iplimentation,
>limit it to about 1200.
>
>> Any sequencing or recovery?
>
>Nope.
>
>> Acknowledgements?
>
>Nope.
>
>> Checksums on or off?
>
>For a local Ethernte segment, you let the MAC layer worry about that.


Well, if that's what people have been doing, then I think the answer
to my original question is:  no, nothing usable is standard(-ish).  I
don't think any usable UDP-based protocol can avoid some error
handling (unless it's a protocol for something where data loss is
tolerable - media streams, for example).  Which doesn't mean that for
some protocols it's not utterly trivial: Didn't get your DNS response?
Send the query again.

I suspect we'll probably end up with a proper TCP, the advantages of
not having to deal with the client are significant, despite the
considerable extra complexity on the device.

Remember that the starting point here is little more than the
equivalent of fgets() or puts() support to a serial port (hardcoded
9600-N-8-1, no flow control).  You can do that (less any actual
command implemented) in under 500 LOCs. The *fancy* version assumes a
VT-100.  Going the TCP route with the Ethernet chip driver, IP, UDP,
DHCP, ICMP, and TCP, and you're talking well over 10KLOCs, with over
half that being in TCP.


>> With UDP (and a bit of consideration at the protocol layer), you can
>> even avoid any IP configuration requirements (at least on a single
>> LAN) by doing everything with broadcasts.
>
>Some IP stacks won't receive UDP broadcast packets with a
>non-reachable source IP.


The whole what-do-you-do-if-you-don't-have-an-IP-config thing is a
PITA for IPv4.  Hope for DHCP, then try a link-local address, then try
a fixed local address, then...  But that's an issue for any
implementation, but UDP offers some additional possibilities (like
broadcasts).  But nothing works everywhere.


>> I've simply never implemented anything with UDP that didn't take error
>> handling into account - is the common practice just to assume that you
>> will not lose/reorder/duplicate/mangle any packing so long as you're
>> on a local segment?
>
>Unless your Ethernet segment is pretty heavily loaded, I find you can
>ignore all that.
>
>If you can't, then you need the equivalent of TCP.

On Wed, 14 Dec 2016 15:22:42 -0800, Paul Rubin
<no.email@nospam.invalid> wrote:

>Robert Wessel <robertwessel2@yahoo.com> writes:
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.
>
>I still don't see the problem: TCP has been around for decades and there
>are some tiny implementations, especially if you don't need multiple
>packets in the air at the same time etc.  Why not just use one?


That may well be the way we go, but given that more than a few
UPD-only implementations have been done, I was trying to determine if
a lighter-weight alternative was feasible.