Reply by Morten Reistad January 15, 20172017-01-15
In article <o2sfro$psh$1@dont-email.me>,
Don Y  <blockedofcourse@foo.invalid> wrote:
>On 12/14/2016 1:03 PM, Robert Wessel wrote: >> On Tue, 13 Dec 2016 12:53:57 -0700, Don Y >> <blockedofcourse@foo.invalid> wrote: >> >>>> OTOH, despite looking, I just don't see any standards for doing that >>>> sort of thing. Am I just missing it? Is there a defacto standard >>>> even if there's no RFC? >>> >>> RLOGIN is much lighter-weight than TELNET (YMMV). But, as with TELNET, >>> runs atop TCP. >> >> Not really sure I'd agree with that, but both are pretty simple. If I >> had TCP, doing either TELNET or RLOGIN (or HTTP) is not much of an >> issue. OTOH, there doesn't really seem to be much advantage for doing >> RLOGIN over TELNET. > >Just to make sure there's no misunderstanding, RLOGIN runs over TCP; >you'd use it in lieu of TELNET. (a full TELNET implementation is >many thousands of lines of code -- much of which you probably won't >need if you just want to emulate a serial port)
Minimal telnet is just sending a handful of bytes as soon as the session is established, and then just use the raw tcp. You would want to turn off the default line-mode, and set character mode. That is just one small printf/echo/print/whatever as soon as you have the session set up. You would need a retransmit timer, but it can be _very_simple, e.g. 6 seconds if you haven't seen an ack. You need to handle four window pointers, one for how far we have sent and one for how far we have received, for both transmit and receive. They are set in the initial syn-ack-synack dialogue. You would need to implement fin-finack-fin2 to terminate the session as well.
>>> What criteria do you place on the operating environment? E.g., you >>> can't run a bare "serial port" 1000km. >> >> No, but one would hope to do something with a bit more range than a >> serial port (which is what's there now), and a bit more convenient >> infrastructure-wise. Design "range" should be something on the >> data-center to campus scale. One of the prime considerations is the >> ability to manage many devices without the complexity of attaching to >> many serial ports scattered around the data center. > >Once you "leave the room", things typically get complicated (unless >you're willing to live with a 100m cable limitation). And, once you >start trying to talk to existing fabric, you have even less leeway >in terms of the corners you can cut (e.g., what happens if some >host tries to ping your node? open an FTP connection? etc -- even >if wrong in attempting to do so). > >Many years ago (10+) I put together a proposal for a low cost "console server" >(essentially a one-port terminal server) to facilitate LOM in a datacenter >environment. At the time, I could enhance security and cut cost by >bastardizing the network protocols with the caveat: all of these devices will >reside on an isolated internet (so they didn't have to contend with the >variety of traffic they might encounter on a "live" internet). > >Nowadays, SoC's have far more resources so even a complete stack isn't >going to break the bank (unless you're *really* scraping the bottom >of the cost/commodity barrel). > >>> Are you just trying to exploit >>> an existing connector (RJ45) for a secondary purpose with a LOCAL >>> "console"? I.e., do you need flow control, fragmentation/packet reassembly, >>> routing, etc.? Or, will you just slap some "special device" on the >>> wire "in close proximity to" the device in question? >> >> I don't really want a special device, although a PC with special > >The "PC" was the intent of my reference :> The point being, it's not >an OTS solution (you still need to "develop" that end of the link). > >> software is reasonable, but at worst case you'd want that PC to need >> to be no closer than somewhere on the local LAN (data center or >> campus, as mentioned above, would be better). If you didn't mind a >> special device, there are RS-232-to-TELNET devices you could just plug >> into the serial port, and define the whole problem away. As to flow >> control and whatnot, those issue should be kept to a minimum (again >> TFTP is a parallel), but since this has been done before, I'm trying >> to figure out if there's existing practice to piggyback on. If, for >> example, there's a semi-standard way to do this, there are likely >> terminal emulators that already support that. > >There have been lots of attempts at making "console servers" (my >reference above). HP had a "Secure Web Console" that presented >a "terminal emulator" in a web browser window that was tethered >to the remote device's serial port. > >[It was a poorly designed product, though -- particularly the >laughable "secure" aspect of it! :> ] > >You can buy single port terminal servers in a package not >much larger than a DB25 backshell (the proposal I outlined above >called for a DB9 backshell package as most servers now have 9 pin >serial ports so you don't have room for a "box" behind the server) > >> The big advantage to going "whole hog" and doing TCP and TELNET (or >> TCP and HTTP), is that you open this up to the whole Internet, and an >> untold number of existing clients for any conceivable platform. > >As Devil's Advocate, I'd point out that you now present a larger >attack surface and a (potentially) bigger adversary pool; one >thing with which a simple serial port didn't have to contend! > >> Physically, the idea would be to replace the serial port with an >> Ethernet port. It would be nice to be able to still connect without >> any network infrastructure at all (beyond, possibly, a crossover >> Ethernet cable). > >It might be interesting to consider putting serial port *and* ethernet >on the same connector (I'd have to consider the hardware consequences >to see if there's a "cheap" way of doing so). > >>> Does the interface have to be interactive? E.g., you could leverage >>> something akin to TFTP to push a "screen" of data to/from a console >>> device (treat the screen as a "form" that is filled in instead of >>> as a STREAM). >> >> The model under consideration is a replacement of a serial console >> assumed to be a VT-100 compatible device. I don't think a hack like >> you're suggesting is viable. At that point we're talking about custom >> software on both ends, so a "proper" TELNET-ish* protocol over UDP >> would be simpler. Developing such a (new) protocol is no problem. > >If you were willing to make the "special device" a PC, then this >just changes the nature of the software running in the PC (given >that the "device end" is already negotiable). > >I offer it only because I've frequently designed "screen oriented" >user interfaces (think: menu/dialog system atop curses(1)) so the >UI is always a ~2KW buffer in the device's memory that just gets >painted (via ANSI escape sequences) onto the hardware TTY. Pushing >those 4KB out through a set of ~8 UDP packets (on demand any time >user asks for the screen to be repainted) would be a piece of cake; >much simpler than the curses stack residing in the device -- or the >ANSI emulator in the PC! > >[The Sun Ray operated in a similar fashion; and, AFAICT, the Pano >cubes (no CPU -- entirely FPGA based) use a similar approach to >"painting a screen"] > >> *And let's not get too hung up on the specifics of TELNET here - some >> bits of handshake information probably need to be exchanged at the top >> of the session, and then it's just a packaging of the bi-directional >> serial stream. > >If you want to interoperate with other devices, you may have to implement >bits of the protocol that you'd hoped you could elide. E.g., MS's >TELNET client is a PITA for me to use, here -- so I install third-party >TELNET clients that "work". > >>> You can also implement a much pared-back network stack by putting >>> other constraints on how you plan to USE that stack (limit number >>> of open sockets, window size, etc.) E.g., my devices are intended to >>> talk to specific *other* devices so I don't have to maintain ARP >>> caches, multiple endpoints, etc. >> >> Sure: I'd like to pare-back the network stack by not implementing TCP. >> ;-) Limited UDP-only on top of Ethernet is pretty trivial. > >But, even with TCP, there are economies available. If you try to >cheap out and port an existing stack (FOSS), you will typically end >up with more resources (and features) than you might otherwise need. > >IMO, the biggest downside is you'll typically insulate yourself from >the inner workings of the stack and, thus, not be aware of the sorts of >exploits that lie in wait. Or, the costs (space/performance) that you >are bearing. > >Thereafter, you have to track new releases of that codebase to see >if/when to fold them into one of your "updates". This could result in >your taking on additional functionality/resource requirements that >you might not want. > >OTOH, if you "take ownership" of such a codebase (i.e., start tweeking >it to *fit* your needs), then you're still stuck watching the releases >and now have to merge "pertinent" changes into your codebase. > >[I.e., you quickly start to feel like the dog's *tail*!] > >If you haven't, already, Comer's XINU (vol2) is a lightweight introduction >to the stacks; followed by his _Internetworking with TCP/IP_ series. >With those behind you, Stevens' _TCP/IP Illustrated_ is by far the best >reference. > >[Though Stevens' texts tend to describe more efficient (from the >standpoint of performance) implementations; Comer's tend to be much >easier to visualize -- more *discrete* layers (and, thus, less >efficient)] > >And, watch the CERT advisories to see how profoundly varied attacks >on stacks can be, in practice!
You would employ such a server behind one or more firewalls, I am sure. Just filter it from anyone that doesn't belong there. This is for console access, not any big user community. -- mrr
Reply by Robert Wessel January 3, 20172017-01-03
On Mon, 26 Dec 2016 23:00:41 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 12/26/2016 3:34 PM, Robert Wessel wrote: >> On Thu, 15 Dec 2016 11:06:19 -0700, Don Y >> <blockedofcourse@foo.invalid> wrote: >> >>> My read of the OP's comments was that he wanted a means of talking with >>> the "command/control console" of his device, remotely, using network >>> fabric instead of a dedicated "serial" link. >>> >>> E.g., in a data center environment, this would be used for LOM -- in >>> lieu of a "display and keyboard". >> >> While it's moot at this point, I need to support more low-level stuff >> than an operational console. Think BIOS setup screens. > >That was the sort of application that inspired my "screen" comment: > "Does the interface have to be interactive? E.g., you could leverage > something akin to TFTP to push a "screen" of data to/from a console > device (treat the screen as a "form" that is filled in instead of > as a STREAM)." > >But, your other requirements suggest you *also* need a stream-oriented >interface. > >While you can layer a *screen* oriented interface atop a stream, the >opposite is a bit tougher. > >When you also throw in the need to transit network segments, it seems >the only practical approach is to implement a pared down TCP and >RLOGIN (or TELNET, depending on the clients you have available).
As I mentioned, it's moot. For devices where we're going to have to support those functions via TCP, we're almost certain to do HTTP. It does mean rewriting the configuration screens handling code, but in many cases that will be less than painful - there's usually already an abstraction layer around the actual VT-100 driver code that implements a panel. Having that output HTML will typically be straight-forward. Even for the devices that only have a command line ("TTY") interface, you can fake that up with HTTP with little effort (OTOH, improving that would likely be a goal).
Reply by Don Y December 27, 20162016-12-27
On 12/26/2016 3:34 PM, Robert Wessel wrote:
> On Thu, 15 Dec 2016 11:06:19 -0700, Don Y > <blockedofcourse@foo.invalid> wrote: > >> My read of the OP's comments was that he wanted a means of talking with >> the "command/control console" of his device, remotely, using network >> fabric instead of a dedicated "serial" link. >> >> E.g., in a data center environment, this would be used for LOM -- in >> lieu of a "display and keyboard". > > While it's moot at this point, I need to support more low-level stuff > than an operational console. Think BIOS setup screens.
That was the sort of application that inspired my "screen" comment: "Does the interface have to be interactive? E.g., you could leverage something akin to TFTP to push a "screen" of data to/from a console device (treat the screen as a "form" that is filled in instead of as a STREAM)." But, your other requirements suggest you *also* need a stream-oriented interface. While you can layer a *screen* oriented interface atop a stream, the opposite is a bit tougher. When you also throw in the need to transit network segments, it seems the only practical approach is to implement a pared down TCP and RLOGIN (or TELNET, depending on the clients you have available).
Reply by December 27, 20162016-12-27
On Mon, 26 Dec 2016 16:34:08 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:

>On Thu, 15 Dec 2016 11:06:19 -0700, Don Y ><blockedofcourse@foo.invalid> wrote: > >>My read of the OP's comments was that he wanted a means of talking with >>the "command/control console" of his device, remotely, using network >>fabric instead of a dedicated "serial" link. >> >>E.g., in a data center environment, this would be used for LOM -- in >>lieu of a "display and keyboard". > > >While it's moot at this point, I need to support more low-level stuff >than an operational console. Think BIOS setup screens.
This is a completely new requirement. No way you could handle this with UDP or even TCP.. You will need some remote KVM system (such as Black Box) which transfer the keyboard, mouse signals as well as the (VGA) scan raster to a remote site. If the data center device can be controlled solely over a serial link, just install an ethernet to serial converter close to the device to be controlled and connect to that converter using standard Telnet.
Reply by Robert Wessel December 26, 20162016-12-26
On Thu, 15 Dec 2016 21:42:42 -0600, Les Cargill
<lcargill99@comcast.com> wrote:

>> An obvious alternative is to implement the "serial console" as a >> TELNET server. Conceptually this is trivial, not to mention well >> standardized and well understood. OTOH, this piles on a fair bit of >> complexity, and requires something akin to a full TCP/IP stack. And >> if you're going to do that, you might as well just go for a web-based >> admin, it's only going to be a bit more work. >> > >A bit??? I think you left a "quite" out. :) If the target >is Linux, then it's less trouble, but it's still somewhat involved.
A basic HTTP server needs surprisingly little. The one in our library is a bit over 2KLOCs (and there's some utility stuff which can expand that), but can be built in iterative, select/state-machine and threaded models, and has a few features not strictly needed. A simple, iterative only, model could probably be squeezed into half that. Obviously it's no threat to Apache's dominance, and the web programming is pretty low level. I cobbled the following up in a couple of minutes: int p_index(void) { int i; time_t rawtime; struct tm * timev; SendHTMLTextHeader(); if (MethodHEAD()) return 1; SendHTMLText( "<html><body>" "<p align=\"center\"><img src=\"c-logo.gif\" width=\"328\" height=\"76\"> </p>" "<h1>Hello world!</h1>" ); time(&rawtime); timev = localtime(&rawtime); SendHTMLprintf("It is: %s</br>", asctime(timev)); SendHTMLprintf("The passed (%i) parameters are:</br><blockquote>", parmcount); for (i=0; i<parmcount; i++) SendHTMLprintf("%s=%s</br>", parms[i].name, parms[i].value); SendHTMLText("</blockquote>"); SendHTMLText("</body></html>"); SendEnd(); return 1; } #include "c-logo.h" int p_c_logo_gif(void) { return SendHTMLBinary(c_logo_gif_data, c_logo_gif_len, "image/gif"); } struct pages_t pages[] = { {"/", p_index}, {"/index.htm", p_index}, {"/index.html", p_index}, {"/c-logo.gif", p_c_logo_gif}, {NULL, NULL} }; int main() { int r; r = WebServer(pages, 2505); printf("rc=%d, msg=%s\n", r, GetMsg()); } (The above would be correct for the iterative version.) The file c-logo.h contains a GIF, a binary file converted to a .h with a trivial utility. Something like: // bin2h file conversion of c-logo.gif unsigned char c_logo_gif_data[] = { /* 0x0000 */ 0x47,0x49,0x46,0x38,0x37,0x61,0x48,0x01, /* 0x0008 */ 0x4c,0x00,0xf7,0x00,0x00,0xff,0xff,0xff, /* 0x0010 */ 0xe7,0xe7,0xe7,0xde,0xde,0xde,0xd6,0xd6, ... /* 0x1700 */ 0x2f,0x11,0x01,0x01,0x00,0x3b, }; long c_logo_gif_len = 5894;
Reply by Robert Wessel December 26, 20162016-12-26
On Thu, 15 Dec 2016 11:06:19 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>My read of the OP's comments was that he wanted a means of talking with >the "command/control console" of his device, remotely, using network >fabric instead of a dedicated "serial" link. > >E.g., in a data center environment, this would be used for LOM -- in >lieu of a "display and keyboard".
While it's moot at this point, I need to support more low-level stuff than an operational console. Think BIOS setup screens.
Reply by Don Y December 25, 20162016-12-25
On 12/25/2016 6:45 AM, Theo Markettos wrote:
> Grant Edwards <invalid@invalid.invalid> wrote: >> One clever method for establishing an initial IP address that I've >> seen is that the embedded device watches for unicast IP packets sent >> to its MAC address. When it sees one, it temporarily adopts whatever >> destination IP address was in the packet. That way, to initially >> configure the device's IP, all you have to do is add an entry to your >> host's ARP cache with the desired IP address and then ping that IP >> address. IIRC, the device in question only did this if the device was >> at factory defaults. > > Or you could just use IPv6: router broadcasts Router Advertisements > containing the local network prefix. Client picks that up, appends its MAC > address, it now has its IP. Everyone else can work out the IP too. > This is not special 'factory defaults' magic, it's how IPv6 works all the > time.
[N.B. IPv6 is a costlier stack] The problem with all discovery/configuration services is they can be spoofed by a malicious host ("server") introduced on the local network. And, unless you have an IDS in place and/or vigilant staff monitoring ALL log files for "unusual" but TRANSIENT (!) error messages ("address already in use", etc.), you're unlikely to know a service has been hijacked to compromise one or more hosts (and wonder the use for which THOSE hosts are now targeting).
Reply by Theo Markettos December 25, 20162016-12-25
Grant Edwards <invalid@invalid.invalid> wrote:
> One clever method for establishing an initial IP address that I've > seen is that the embedded device watches for unicast IP packets sent > to its MAC address. When it sees one, it temporarily adopts whatever > destination IP address was in the packet. That way, to initially > configure the device's IP, all you have to do is add an entry to your > host's ARP cache with the desired IP address and then ping that IP > address. IIRC, the device in question only did this if the device was > at factory defaults.
Or you could just use IPv6: router broadcasts Router Advertisements containing the local network prefix. Client picks that up, appends its MAC address, it now has its IP. Everyone else can work out the IP too. This is not special 'factory defaults' magic, it's how IPv6 works all the time. Theo
Reply by Grant Edwards December 24, 20162016-12-24
On 2016-12-24, Morten Reistad <first@last.name.invalid> wrote:

> You can also have a very rudimentary telnet negotiation, basically > enforcing a standard set of options and accepting nothing else.
That turns out to be surprisingly difficult. It's not a lot of code, but no two clients behave the same, and the first time you trie a new client, you find a new problem.
>>Yet what exactly is the protocol? How many characters can you put in >>a packet? Any sequencing or recovery? Acknowledgements? Checksums >>on or off? With UDP (and a bit of consideration at the protocol >>layer), you can even avoid any IP configuration requirements (at least >>on a single LAN) by doing everything with broadcasts. > > You probably have to have some default IP address, which you can > change. Making a simple udp one-shot protocol to set such options > should be easy.
One clever method for establishing an initial IP address that I've seen is that the embedded device watches for unicast IP packets sent to its MAC address. When it sees one, it temporarily adopts whatever destination IP address was in the packet. That way, to initially configure the device's IP, all you have to do is add an entry to your host's ARP cache with the desired IP address and then ping that IP address. IIRC, the device in question only did this if the device was at factory defaults. -- Grant
Reply by Morten Reistad December 24, 20162016-12-24
In article <qs535ctbk361kav29rjd236dat22mo199e@4ax.com>,
Robert Wessel  <robertwessel2@yahoo.com> wrote:
>On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards ><invalid@invalid.invalid> wrote: > >>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote: >> >>> OTOH, it's clear that a number of people have implemented a UDP-based >>> protocol for handling the sort of things often done with a serial >>> console attached to a device. This has a number of advantages - you >>> can fake UPD over Ethernet with very little code, especially if you >>> define away things like fragmentation and options. A simplified >>> TELNET-ish protocol on top of that would be pretty simple, you just >>> need some basic packet sequencing and acknowledgements. >> >>I know of people who have implemented that on top of IP rather than >>UDP. They call it TCP. ;) > > >Well, yes, but the point is that TCP is a fair chunk of code, and for >this function it has to go in a spot where it's less than convenient >to update. Hence the desire for maximum simplicity. I used TFTP as >an example, since it solves a similar problem (and you could >certatinly trivially implemented a TCP version of TFTP), with similar >constraints.
A rudimentary "simple-tcp" that is not quite compliant with TCP but will interoperate with one that is is not much code. You will have to handle a connection or maybe a handful, but not very many. You will have to do retransmissions, but you can back off to a constant, semi-high value (like 6 seconds) on the second attempt. You can deny all options, and stay with a small window of a few k, and send 512 bytes or less. That should reduce the requirements to a few hundred lines of code beyond implementing UDP. You can also have a very rudimentary telnet negotiation, basically enforcing a standard set of options and accepting nothing else.
>>> IOW, something with a relationship to TELNET sort of like TFTPs >>> relationship to FTP. >> >>Just use a raw TCP connection. There's no need to implement the TELNET >>protocol at all. The Unix command line client will skip all the >>handshaking stuff if you use a non-standard port. Or you can use >>netcat. Or any number of terminal emulators or COM port redirectors >>that support raw TCP connections. > > >Given how little TELNET actually does (or at least, needs to do), if >you have TCP, there would be no really issue implementing that. Raw >TCP is obviously an option, but again, the incremental cost to do >actual TELNET is low. And as I said, if you have TCP, why not just do >this with HTTP, and really make things convenient for people. > > >>If you're on a local Ethernet segment, just use raw UDP. I've been >>using both those approaches for close to 20 years, and have never had >>any problems. > > >Yet what exactly is the protocol? How many characters can you put in >a packet? Any sequencing or recovery? Acknowledgements? Checksums >on or off? With UDP (and a bit of consideration at the protocol >layer), you can even avoid any IP configuration requirements (at least >on a single LAN) by doing everything with broadcasts.
You probably have to have some default IP address, which you can change. Making a simple udp one-shot protocol to set such options should be easy.
>I've simply never implemented anything with UDP that didn't take error >handling into account - is the common practice just to assume that you >will not lose/reorder/duplicate/mangle any packing so long as you're >on a local segment? Again, TFTP addresses those issues, at least as >they apply to its requirements. > > >>> OTOH, despite looking, I just don't see any standards for doing that >>> sort of thing. Am I just missing it? Is there a defacto standard >>> even if there's no RFC? >> >>If you want something reliable that works over WANs, use TCP: it's the >>only way you're going to dependably make anything work through a >>firewall. > > >Sure, no disagreement there. And no, operation over WANs is not a >prime consideration, although it certainly wouldn't hurt if it could, >even if the user had to deal with firewall issues, or set up a relay >server of some sort. But the scope would be within a data-center, or >perhaps campus (assuming at least real IP address assignment - without >that, local LAN). Remember the idea here is to replace a serial >console port.
A "simple-TCP" will handle this, and you could make a udp simple config-setter like the ones digium does for the iaxys should be an afternoon's work.
>>If you're going to assume local Ethernet, you can just use UDP.
You will be amazed on how much chatter, duplications and other stuff I have found on ethernets. -- mrr