On 2014-01-09, upsidedown@downunder.com <upsidedown@downunder.com> wrote:> On Wed, 8 Jan 2014 22:14:35 +0000 (UTC), Grant Edwards ><invalid@invalid.invalid> wrote: > >>On 2014-01-08, Aleksandar Kuktin <akuktin@gmail.com> wrote: >> >>> I'm making a protocol for communication between a PC and a peripheral >>> device. The protocol is expected to, at first, run on raw Ethernet >> >>I've been supporting a protocol like that for many years. Doing raw >>Ethernet on Windows hosts is becoming increasingly problematic due to >>attempts by Microsoft to fix security issues. We anticipate it will >>soon no longer be feasible and we'll be forced to switch to UDP. > > UDP adds very little compared to raw ethernet, some more or less > stable header bytes and a small ARP protocol (much less than a page of > code). There are a lot of tools to display the various IP and UDP > headers and standard socket drivers should work OK. > > If you are using raw ethernet on a big host, you most likely would > have to put the ethernet adapter into promiscuous mode, which might > security / permission issue.I've never found that to be the case. However raw ethernet access in non-promiscuous still requires admin/root/special privledges and causes a lot of security headaches (particularly under Windows). -- Grant Edwards grant.b.edwards Yow! I'm continually AMAZED at at th'breathtaking effects gmail.com of WIND EROSION!!
Binary protocol design: TLV, LTV, or else?
Started by ●January 8, 2014
Reply by ●January 9, 20142014-01-09
Reply by ●January 9, 20142014-01-09
On Thu, 09 Jan 2014 07:05:33 -0700, Don Y <this@isnotme.com> wrote:>Hi Dimiter, > >On 1/9/2014 1:53 AM, dp wrote: >> On Thursday, January 9, 2014 10:28:21 AM UTC+2, Don Y wrote: >>> ... >>> I've (we've?) been assuming he can cram an entire message into >>> a tiny "no-fragment" packet -- that may not be the case! (Or, >>> may prove to be a problem when run over protocols with smaller >>> MTU's) >> >> UDP does not add any fragmentation overhead compared to his >> raw Ethernet anyway (that is, if he stays with UDP packets >> fitting in apr. 1500 bytes he will be no worse off than without >> UDP).I have been running raw Ethernet since the DIX days, with DECnet, LAT (similar to "telnet" terminal connections) and my own protocols forcing the network adapters into promiscious mode on thick Ethernet cables with vampire taps on the cable.>I'm thinking more in terms of any other media (protocols) over >which he may eventually use for transport. If he doesn't want to >add support for packet reassembly in *his* protocol, then he >would be wise to pick a message format that fits in the smallest >MTU "imaginable".While some X.25 based protocols might limit the frame size to 64 bytes, 576 bytes has been the norm for quite a few years. Standard Ethernet frames are above 1400 bytes, while Jumbo frames could be about 9000 bytes.>For ethernet, I think that is ~60+ octets (i.e., just bigger than the >frame header).64 bytes is the minimum for proper collision detection size on coaxial Ethernet networks.
Reply by ●January 9, 20142014-01-09
On 1/9/2014 2:55 PM, upsidedown@downunder.com wrote:> On Thu, 09 Jan 2014 07:05:33 -0700, Don Y<this@isnotme.com> wrote: > >> Hi Dimiter, >> >> On 1/9/2014 1:53 AM, dp wrote: >>> On Thursday, January 9, 2014 10:28:21 AM UTC+2, Don Y wrote: >>>> ... >>>> I've (we've?) been assuming he can cram an entire message into >>>> a tiny "no-fragment" packet -- that may not be the case! (Or, >>>> may prove to be a problem when run over protocols with smaller >>>> MTU's) >>> >>> UDP does not add any fragmentation overhead compared to his >>> raw Ethernet anyway (that is, if he stays with UDP packets >>> fitting in apr. 1500 bytes he will be no worse off than without >>> UDP). > > I have been running raw Ethernet since the DIX days, with DECnet, LAT > (similar to "telnet" terminal connections) and my own protocols > forcing the network adapters into promiscious mode on thick Ethernet > cables with vampire taps on the cable. > >> I'm thinking more in terms of any other media (protocols) over >> which he may eventually use for transport. If he doesn't want to >> add support for packet reassembly in *his* protocol, then he >> would be wise to pick a message format that fits in the smallest >> MTU "imaginable". > > While some X.25 based protocols might limit the frame size to 64Anything in the chain can set the MTU to 68 bytes and still be "playing by the rules". So, if you *rely* on 70 octets coming down the 'pike in one UNFRAGMENTED datagram, if your PMTUd gives something less, you won't receive that level of service.> bytes, 576 bytes has been the norm for quite a few years. > Standard Ethernet frames are above 1400 bytes, while Jumbo frames > could be about 9000 bytes. > >> For ethernet, I think that is ~60+ octets (i.e., just bigger than the >> frame header). > > 64 bytes is the minimum for proper collision detection size on > coaxial Ethernet networks.From RFC791: "Every internet module must be able to forward a datagram of 68 octets without further fragmentation. This is because an internet header may be up to 60 octets, and the minimum fragment is 8 octets." "Every internet destination must be able to receive a datagram of 576 octets either in one piece or in fragments to be reassembled." So, a datagram could, conceivably, be fragmented into hundreds of 68 octet datagrams (which can include padding). Yet, must be able to reassemble these to form that original datagram. I.e., I could build a bridge that diced up incoming datagrams into itsy bitsy pieces and be strictly compliant -- as long as I could handle a 576 octet datagram (buffer size). OTOH, reliable PMTU discovery is problematic on generic networks as many nodes don't handle (all) ICMP traffic (as originally intended). But, nothing requires the nodes/hops to handle a ~1500 octet datagram ("Datagram Too Big") Folks working on big(ger) iron often don't see where all the dark corners of the protocols manifest themselves. And, folks writing stacks often don't realize how much leeway they actually have in their implementation(s)! :< [N.B. IPv6 increases these numbers]
Reply by ●January 9, 20142014-01-09
On 2014-01-09, Don Y <this@isnotme.com> wrote:> Anything in the chain can set the MTU to 68 bytes and still be > "playing by the rules". So, if you *rely* on 70 octets coming > down the 'pike in one UNFRAGMENTED datagram, if your PMTUd gives > something less, you won't receive that level of service.Let's not forget that we're discussing UDP _as_a_substitute_for_ _raw_Ethernet_. That means the OP is willing to require that the two nodes are on the same network segment, and that we can assume that an Ethernet frame of 1500 bytes is OK. If using UDP allows packets to be routed between two remote nodes _some_ of the time, that's still pure gravy compared to using raw Ethernet -- even if the UDP/IP implementation doesn't support fragmentation. -- Grant Edwards grant.b.edwards Yow! PEGGY FLEMMING is at stealing BASKET BALLS to gmail.com feed the babies in VERMONT.
Reply by ●January 9, 20142014-01-09
Hi Grant, On 1/9/2014 3:44 PM, Grant Edwards wrote:> On 2014-01-09, Don Y<this@isnotme.com> wrote: > >> Anything in the chain can set the MTU to 68 bytes and still be >> "playing by the rules". So, if you *rely* on 70 octets coming >> down the 'pike in one UNFRAGMENTED datagram, if your PMTUd gives >> something less, you won't receive that level of service. > > Let's not forget that we're discussing UDP _as_a_substitute_for_ > _raw_Ethernet_. That means the OP is willing to require that the two > nodes are on the same network segment, and that we can assume that an > Ethernet frame of 1500 bytes is OK. > > If using UDP allows packets to be routed between two remote nodes > _some_ of the time, that's still pure gravy compared to using raw > Ethernet -- even if the UDP/IP implementation doesn't support > fragmentation.As I said in my reply to Dimiter, upthread: I'm thinking more in terms of any other media (protocols) over which he may eventually use for transport. Given that the OP is in the process of designing a protocol, he may want to consider the inevitability (?) of his interconnect medium (and/or underlying protocol) changing in the future. CAN-bus, zigbee, etc. I.e., *expecting* to be able to push a 1500 byte message "in one burst" can lead to problems down the road when/if that assumption can no longer be met. Too often, an ignorance of the underlying protocol ends up having disproportionate costs for "tiny" bits of protocol overhead. E.g., adding a header that brings the payload to one byte beyond the MSS. "Why is everything so much slower than it (calculated) should be?" I try to design with a mantra of "expect the least, enforce the most". [The OP hasn't really indicated what sort of environment he expects to operate within nor the intent of the device and the relative importance (or lack thereof) of the comms therein]
Reply by ●January 10, 20142014-01-10
On Wed, 08 Jan 2014 23:09:00 +0000, David LaRue wrote:> Hello, > > I originated a product that used TLV packets back in the 90s and it is > still in use today without any problems. It was similar to a > configuration file that contained various parameters for applications > that shared data. There was a root packet header. This allowed > transmission acros TCP, serial, queued pipes, and file storage. We > enforced a 4-byte alignment on fields due to the machines being used to > parse the data - we had Windows, linux, and embedded devices reading the > data. Just be sure to define the byte order. We wrote and maintained > an RFC like document. > > One rule we followed that may help you is that once a tag is defined it > is never redefined. That prevented issues migrating forward and > backward. Tags could be removed from use, but were always supported. > > One issue we had with TLV was with one of the developers taking > shortcuts. The TLVs were built in a tree so any V started with a TL > until you got to the lowest level item being communicated. Anyway the > developer in question would read the T and presume they could bypass > reading the lower level tags because the order was fixed - it was not. > Upgraded protocols added fields (a low level TLV) that cause read > issues. Easy to find but frustrating that we had to re-release one of > the node devices. > > The only other error you are likely to get is due with TLVs like this is > an issue if they entrire message isn't delivered. The follow on data > becomes part of the previous message. That is why some encaptulation > might be wise. If you are using UDP and there is no need for multiple > packets per message (ever) that might be your encapsulation method. > > Good luck, > > DavidWell, this is reassuring. It means at least someone did what I intend to do, so I should be able to do the same.
Reply by ●January 10, 20142014-01-10
On Wed, 08 Jan 2014 15:19:30 -0700, Don Y wrote:> Hi Aleksander, > > On 1/8/2014 2:30 PM, Aleksandar Kuktin wrote: > >> I'm making a protocol for communication between a PC and a peripheral > > Here there be dragons... > >> device.Will give more details in a follow-up in a different sub-thread.> Are you sure you have enough variety to merit the extra overhead (in the > packet *and* in the parsing of the packet)?Pretty sure. The packet transmited over the wire is actually expected to be an amalgamation of various commands, parameters and options.> Can you, > instead, create a single packet format whose contents are indicated by a > "packet type" specified in the header? Even if this means leaving space > for values/parameters that might not be required in every packet type? > For example: > <header> <field1> <field2> <field3> <field4> > Where certain fields may not be used in certain packet types (their > contents then being "don't care"). > > Alternatively, a packet type that implicitly *defines* the format of the > balance of the packet. For example: > type1: <header1> <fieldA> <fieldB> > type2: <header2> <fieldA> > type3: <header3> <fieldA> <fieldB> <fieldC> <fieldD> > (where the format of each field may vary significantly between message > types)This is explicitly what I don't want. That way, I would need to send many, many packets to transmit my message across.> It seems like you are headed in the direction of: > <header> <fields> > where the number of fields can vary as can their individual formats.It seems this is what I will end up with.>> But I would like to hear other (read: opposing) opinions. Particularly >> so since I am self-taught so there may be considerations obvious to >> graduated engineers that I am oblivious to. >> >> BTW, the periphery that is on the other end is autonomous and rather >> intelligent, but very resource constrained. Really, resource >> constrainment of the periphery is my main problem here. > > So, the less "thinking" (i.e., handling of variations) that the remote > device has to handle, the better.Hmmm... Not really. Avability of CPU cycles depends on other details of the device, but if need be I can make the device drown in its own CPU cycles. Memory, on the other hand, is constrained.> Of course, this can be done in a variety of different ways! > E.g., you could adopt a format where each field consists of: > <parameterNumber> <parameterValue> > and the receiving device can blindly parse the parameterNumber and plug > the corresponding parameterValue into a "slot" in an array of parameters > that your algorithms use. > > Alternatively, you could write a parser that expects an entire message > to have a fixed format and plug the parameters it discovers into > predefined locations in your app.I now go to the other sub-thread to continue the conversation...
Reply by ●January 10, 20142014-01-10
On Wed, 08 Jan 2014 22:14:35 +0000, Grant Edwards wrote:> On 2014-01-08, Aleksandar Kuktin <akuktin@gmail.com> wrote: > >> I'm making a protocol for communication between a PC and a peripheral >> device. The protocol is expected to, at first, run on raw Ethernet > > I've been supporting a protocol like that for many years. Doing raw > Ethernet on Windows hosts is becoming increasingly problematic due to > attempts by Microsoft to fix security issues. We anticipate it will soon > no longer be feasible and we'll be forced to switch to UDP. > > I'm not the Windows guy, but as I understand it you'll have to write a > Windows kernel-mode driver to support your protcol, and users will > require admin privlidges. Even then you'll have problems with various > firewall setups and anti-virus software.TBH, I really don't expect to support Windows, at least for the time being. My reasoning is that I can always patch together a Linux LiveCD and ship it with the device. I began honing my skills with the Linux from scratch project, so assembling a distro should not take me more than a week.> If the PC is running Linux, raw Ethernet isn't nearly as problematic as > it is on Windows, but it does still require either root privledges or > special security capabilities.The idea is to use one program that runs as root and relays packets and have a different program do the actual driving of the device.> If you can, I'd recommend using UDP (which is fairly low overhead). The > PC end can then be written as a normal user-space application that > doesn't require admin privledges. You'll still have problems with some > routers and NAT firewalls, but way fewer problems than trying to use raw > Ethernet.UDP/IP is just an extension of IP. I considered using raw IP, but decided against it on grounds that I didn't want to implement IP, simple as it may be. Ofcourse, I eventually *will* implement IP so then I might end up with the whole UDP/IP but, honestly, at this moment the only benefit of UDP/IP is the ease of writing the driver. But that is a very marginal benefit.> Using TCP will allow the easiest deployment, but TCP requires quite a > bit more overhead than UDP.TCP/IP is out of the question, period.
Reply by ●January 10, 20142014-01-10
On Thu, 09 Jan 2014 07:05:33 -0700, Don Y wrote:> It must have been entertaining for the folks who came up with ethernet, > IP, etc. way back when to start with a clean slate and *guess* as to > what would work best! :>Actually, that's not how it happened at all. :) Just like in any evolutionary process, several possible solutions were produced and the ones that were "fittest" and most adapted to the environment were the ones that prevailed.
Reply by ●January 10, 20142014-01-10
On Thu, 09 Jan 2014 16:56:49 -0700, Don Y wrote:> [The OP hasn't really indicated what sort of environment he expects to > operate within nor the intent of the device and the relative importance > (or lack thereof) of the comms therein]The device is a CNC robot, to be used in manufacture. Because of that, I can require and assume a fairly strict, secure and "proper" setup, with or without Stuxnet and its ilk. The protocol is supposed to support transfer of compiled G-code from the PC to a device (really a battery of devices), transfer of telemetry, configuration and perhaps a few other things I forgot to think of by now. Since its main purpose is transfer of G-code, the protocol is expected to be able to utilize fairly small packets, small enough that fragmentation is not expected to happen (60 octets should be enough).
