EmbeddedRelated.com
Forums

Binary protocol design: TLV, LTV, or else?

Started by Aleksandar Kuktin January 8, 2014
On 1/13/2014 11:23 PM, upsidedown@downunder.com wrote:
> On Mon, 13 Jan 2014 00:30:38 -0700, Don Y<this@isnotme.com> wrote: > >>> Since branches are not allowed in 10Base2, you have to run the bus via >>> _all_ devices, one cable to the T-connector and an other cable back, >>> quickly extending past the 200 m limit. >> >> I don't design aircraft carriers! :> 10m is more than enough to >> run from one end of a piece of equipment to the other -- stopping >> at each device along the way. 10Base2 was a win when you had lots of >> devices "lined up in a row" where it was intuitive to just "daisy >> chain" them together. E.g., imagine what a CAN bus deployment >> would look like if it had to adhere to a physical star topology >> (all those "nodes" sitting within inches of each other yet unable to >> take advantage of their proximity for cabling economies -- instead, >> having to run individual drops off to some central "hub/switch") >> >> [As we were rolling our own hardware, no need for T's -- two BNC's >> on each device: upstream + downstream.] > > Did you by chance have one male and one female BNC on the device ? > > And what happens, when someone wants to pull out the device ? _All_ > traffic in the net is disrupted, until the person finds how to join > the two cables together, i.e. finds a female-female adapter after a 15 > minute search :-)
Its a subassembly. You can't operate it without all the components in place and operational. So, if you want to remove a sensor/actuator, the entire subassembly is down for the duration (like replacing *one* wheel shoe on a car -- impractical to drive it in that condition!) Once the "component" has been replaced and reinstalled on the network segment, the entire subassembly can be brought on-line, again. How do you move air if the actuator that runs the motor is broken and removed from service? How do you measure the airflow if the sensor that monitors it is broken and removed from service? How do you heat the air if the actuator that applies heat is missing? Sense the temperature if the sensor that monitors it is missing? Etc. Never a need to "patch around" a missing/removed component.
On Mon, 13 Jan 2014 18:00:09 -0700, Don Y <this@isnotme.com> wrote:

>Hi Tom, > >On 1/13/2014 5:46 PM, Tom Gardner wrote: >> On 14/01/14 00:35, Don Y wrote: >>> On 1/13/2014 4:21 PM, Robert Wessel wrote: >>> >>>>>> reliable ... but that got fixed and TR's predictable timing made >>>>>> analyzing systems and programming reliably timed delivery - >>>>>> particularly across repeaters - easier even than on CAN. >>>>> >>>>> At one time, I did an analysis that suggested even 4Mb TR would >>>>> outperform 10Mb ethernet when you were concerned with temporal >>>>> guarantees. >>>> >>>> Which was one of the touted features of TRN. Unfortunately for TRN, >>>> approximately zero users actually cared about that. >>> >>> It's too bad that "fast" has won out over "predictable" (in >>> many things -- not just network technology). >> >> Token rings, e.g. FDDI and others, had config/management >> problems that largely negated predictability guarantees, >> e.g. dropped tokens, duplicated tokens, complexity (have >> a look at all the FSMs!) >> >> CSMA/CD is much easier to manage and faultfind. > >The problem is you have to layer a *different* protocol onto those >media if you want deterministic behavior. AND, prevent any >"noncompliant" traffic from using the medium at the same time. > >E.g., you could have "office equipment" and "process control >equipment" sharing a token-passing network and *still* have >guarantees for the process control subsystems. Not the case >with things like ethernet (unless you create a special >protocol stack for those devices and/or interpose some bit of >kit that forces them to "behave" properly).
Horrible idea of putting office Windows machines in the same network as some real process control. These days firewalls are used between the networks and often even special gateways in a DMZ.
On 1/13/2014 11:41 PM, Robert Wessel wrote:
> On Mon, 13 Jan 2014 22:49:53 -0700, Don Y<this@isnotme.com> wrote: > >> Hi Les, >> >> On 1/13/2014 9:40 PM, Les Cargill wrote: >>> Don Y wrote: >> >>>>>>>>> reliable ... but that got fixed and TR's predictable timing made >>>>>>>>> analyzing systems and programming reliably timed delivery - >>>>>>>>> particularly across repeaters - easier even than on CAN. >>>>>>>> >>>>>>>> At one time, I did an analysis that suggested even 4Mb TR would >>>>>>>> outperform 10Mb ethernet when you were concerned with temporal >>>>>>>> guarantees. >>>>>>> >>>>>>> Which was one of the touted features of TRN. Unfortunately for TRN, >>>>>>> approximately zero users actually cared about that. >>>>>> >>>>>> It's too bad that "fast" has won out over "predictable" (in >>>>>> many things -- not just network technology). >>>>> >>>>> Token rings, e.g. FDDI and others, had config/management >>>>> problems that largely negated predictability guarantees, >>>>> e.g. dropped tokens, duplicated tokens, complexity (have >>>>> a look at all the FSMs!) >>>>> >>>>> CSMA/CD is much easier to manage and faultfind. >>>> >>>> The problem is you have to layer a *different* protocol onto those >>>> media if you want deterministic behavior. AND, prevent any >>>> "noncompliant" traffic from using the medium at the same time. >>>> >>>> E.g., you could have "office equipment" and "process control >>>> equipment" sharing a token-passing network and *still* have >>>> guarantees for the process control subsystems. Not the case >>>> with things like ethernet (unless you create a special >>>> protocol stack for those devices and/or interpose some bit of >>>> kit that forces them to "behave" properly). >>> >>> As you are no doubt aware, what happened there was Ethernet >>> switching and it well and truly solved the collision problem. >> >> It's not collisions that are the problem. Rather, it is >> timeliness guarantees. A node on an ethernet switch has >> no guarantee as to when -- or *if* -- its packets will >> be delivered. >> >> Switches have fixed size memories. There are no guarantees that >> packets sent to the switch ever *go* anywhere. >> >> By contrast, token passing networks gave assigned timeslots. >> You *knew* when your "turn" to use the media would come along. >> And, were *guaranteed* this by the basic design of the network >> itself (not some other protocol layered on top of it). > > Even without errors, the constraints weren't very tight - your node > could well get the next available slot after the 200 other nodes > waiting to transmit a 4K* frame, even with priority reservations > (admittedly that could be limited by controlling the number of nodes > on the ring). That would be the better part of two seconds. And if > there was any sort of token recovery action going on, several seconds > of disruption were normal.
Repeat the exercise with *shared* fabric scaled back to 10Mb speeds (to compare fairly with 4Mb TR). Or, switched fabric. How big will the packet buffer need to be in that switch? What admission policies will it exhibit? How will a node know a priori how long it must wait to be *sure* the message it sends will get delivered? What happens if one (or 199!) of the other nodes decides it has "a lot more to say" while the node is waiting for its packet to be delivered? Will those other nodes *appear* to be conspiring to lock out that node? Scale TR up to today's speeds. Pick a comparable ethernet switch and try to come up with a definitive answer. Without being able to characterize the rest of the traffic on that network... I.e., token passing strategies inherently implement a concept of fairness and equal access. You have to *hope* your ethernet switch does -- and try to get the vendor to provide you with quantitative details of its (current!) implementation. Then, pray they never discontinue it! *And* hope the rest of the nodes on YOUR network behave equitably. [Getting details of switch internals is sort of like asking The Colonel for his "secret recipe" [1]] I've been through this exercise before. Getting deterministic behavior from ethernet -- WITHOUT PROTOCOL MODIFICATIONS -- is a real hassle. Witness the assortment of automation protocols "over ethernet"...
> *4472 bytes for 4Mb TRN, 17800 for 16Mb
[1] Actually, I think an analysis of it was done in The Straight Dope or one of Poundstone's books... short answer: "nothing special"
On 1/14/2014 12:04 AM, upsidedown@downunder.com wrote:
> On Mon, 13 Jan 2014 18:00:09 -0700, Don Y<this@isnotme.com> wrote:
>> The problem is you have to layer a *different* protocol onto those >> media if you want deterministic behavior. AND, prevent any >> "noncompliant" traffic from using the medium at the same time. >> >> E.g., you could have "office equipment" and "process control >> equipment" sharing a token-passing network and *still* have >> guarantees for the process control subsystems. Not the case >> with things like ethernet (unless you create a special >> protocol stack for those devices and/or interpose some bit of >> kit that forces them to "behave" properly). > > Horrible idea of putting office Windows machines in the same network > as some real process control. These days firewalls are used between > the networks and often even special gateways in a DMZ.
I don't see a reference to "Windows" anywhere in the above... Regardless, it doesn't change the idea conveyed.
On 14/01/14 01:00, Don Y wrote:
> Hi Tom, > > On 1/13/2014 5:46 PM, Tom Gardner wrote: >> On 14/01/14 00:35, Don Y wrote: >>> On 1/13/2014 4:21 PM, Robert Wessel wrote: >>> >>>>>> reliable ... but that got fixed and TR's predictable timing made >>>>>> analyzing systems and programming reliably timed delivery - >>>>>> particularly across repeaters - easier even than on CAN. >>>>> >>>>> At one time, I did an analysis that suggested even 4Mb TR would >>>>> outperform 10Mb ethernet when you were concerned with temporal >>>>> guarantees. >>>> >>>> Which was one of the touted features of TRN. Unfortunately for TRN, >>>> approximately zero users actually cared about that. >>> >>> It's too bad that "fast" has won out over "predictable" (in >>> many things -- not just network technology). >> >> Token rings, e.g. FDDI and others, had config/management >> problems that largely negated predictability guarantees, >> e.g. dropped tokens, duplicated tokens, complexity (have >> a look at all the FSMs!) >> >> CSMA/CD is much easier to manage and faultfind. > > The problem is you have to layer a *different* protocol onto those > media if you want deterministic behavior. AND, prevent any > "noncompliant" traffic from using the medium at the same time. > > E.g., you could have "office equipment" and "process control > equipment" sharing a token-passing network and *still* have > guarantees for the process control subsystems. Not the case > with things like ethernet (unless you create a special > protocol stack for those devices and/or interpose some bit of > kit that forces them to "behave" properly).
You are stating the obvious "swings", and not mentioning the "roundabouts". If you want *guaranteed* behaviour, you have to consider: - What are the guarantees if a node inserts a second token? - What are the guarantees the token gets dropped? And, of course, there are other failure modes that are more of an issue in token rings, e.g. partitioning, subtle incompatibility between different vendor's protocol stacks etc. In practice CSMA/CD is usually more robust than token ring. Sure CSMA/CD has well-understood limitations which need to be worked around. But the workarounds for token ring deficiencies are very similar to those for CSMA/CD deficiencies, so there's no added disadvantage to just using CSMA/CD.
Don Y wrote:
> Hi Robert, > > On 1/13/2014 9:55 PM, Robert Wessel wrote: > > [attrs elided] > >>>> The problem is you have to layer a *different* protocol onto those >>>> media if you want deterministic behavior. AND, prevent any >>>> "noncompliant" traffic from using the medium at the same time. >>>> >>>> E.g., you could have "office equipment" and "process control >>>> equipment" sharing a token-passing network and *still* have >>>> guarantees for the process control subsystems. Not the case >>>> with things like ethernet (unless you create a special >>>> protocol stack for those devices and/or interpose some bit of >>>> kit that forces them to "behave" properly). >>> >>> As you are no doubt aware, what happened there was Ethernet >>> switching and it well and truly solved the collision problem. >>> >>> Most, if not all packets live on a collision domain with exactly >>> two NICs on it - except for 802.11>x< , where just about >>> any concept smuggled form the other old standards doubtless lives in >>> the air link. >> >> And given that most 100Mb and faster (wired) Ethernet links are full >> duplex these days, there's effectively no collision domain at all. >> >> OTOH, that doesn't prevent the switch from dropping packets if the >> destination port is sufficiently busy. > > Or for the actions of one node influencing the delivery of traffic > from *another* node! ("Betty in accounting is printing a lengthy > report -- the CNC machines have stopped as their input buffers are > now empty...")
So VLANs... and 802.11Q or other CoS/QoS .... Or hie down t' the Wally World and buy a Netgear switch or two and leave Betty's print job alone ... -- Les Cargill
On Tue, 14 Jan 2014 03:23:41 -0700, Don Y <This.is@not.Me> wrote:

>On 1/13/2014 11:41 PM, Robert Wessel wrote: >> On Mon, 13 Jan 2014 22:49:53 -0700, Don Y<this@isnotme.com> wrote: >> >>> Hi Les, >>> >>> On 1/13/2014 9:40 PM, Les Cargill wrote: >>>> Don Y wrote: >>> >>>>>>>>>> reliable ... but that got fixed and TR's predictable timing made >>>>>>>>>> analyzing systems and programming reliably timed delivery - >>>>>>>>>> particularly across repeaters - easier even than on CAN. >>>>>>>>> >>>>>>>>> At one time, I did an analysis that suggested even 4Mb TR would >>>>>>>>> outperform 10Mb ethernet when you were concerned with temporal >>>>>>>>> guarantees. >>>>>>>> >>>>>>>> Which was one of the touted features of TRN. Unfortunately for TRN, >>>>>>>> approximately zero users actually cared about that. >>>>>>> >>>>>>> It's too bad that "fast" has won out over "predictable" (in >>>>>>> many things -- not just network technology). >>>>>> >>>>>> Token rings, e.g. FDDI and others, had config/management >>>>>> problems that largely negated predictability guarantees, >>>>>> e.g. dropped tokens, duplicated tokens, complexity (have >>>>>> a look at all the FSMs!) >>>>>> >>>>>> CSMA/CD is much easier to manage and faultfind. >>>>> >>>>> The problem is you have to layer a *different* protocol onto those >>>>> media if you want deterministic behavior. AND, prevent any >>>>> "noncompliant" traffic from using the medium at the same time. >>>>> >>>>> E.g., you could have "office equipment" and "process control >>>>> equipment" sharing a token-passing network and *still* have >>>>> guarantees for the process control subsystems. Not the case >>>>> with things like ethernet (unless you create a special >>>>> protocol stack for those devices and/or interpose some bit of >>>>> kit that forces them to "behave" properly). >>>> >>>> As you are no doubt aware, what happened there was Ethernet >>>> switching and it well and truly solved the collision problem. >>> >>> It's not collisions that are the problem. Rather, it is >>> timeliness guarantees. A node on an ethernet switch has >>> no guarantee as to when -- or *if* -- its packets will >>> be delivered. >>> >>> Switches have fixed size memories. There are no guarantees that >>> packets sent to the switch ever *go* anywhere. >>> >>> By contrast, token passing networks gave assigned timeslots. >>> You *knew* when your "turn" to use the media would come along. >>> And, were *guaranteed* this by the basic design of the network >>> itself (not some other protocol layered on top of it). >> >> Even without errors, the constraints weren't very tight - your node >> could well get the next available slot after the 200 other nodes >> waiting to transmit a 4K* frame, even with priority reservations >> (admittedly that could be limited by controlling the number of nodes >> on the ring). That would be the better part of two seconds. And if >> there was any sort of token recovery action going on, several seconds >> of disruption were normal. > >Repeat the exercise with *shared* fabric scaled back to 10Mb speeds >(to compare fairly with 4Mb TR). Or, switched fabric. How big will >the packet buffer need to be in that switch? What admission policies >will it exhibit? How will a node know a priori how long it must wait >to be *sure* the message it sends will get delivered? What happens if >one (or 199!) of the other nodes decides it has "a lot more to say" >while the node is waiting for its packet to be delivered? Will those >other nodes *appear* to be conspiring to lock out that node?
I didn't say that TRN didn't provide stronger guarantees than Ethernet (it did), rather that the guarantees were sufficiently weak that they were mostly useless.
>Scale TR up to today's speeds. Pick a comparable ethernet switch and >try to come up with a definitive answer. Without being able to >characterize the rest of the traffic on that network...
TRN was going switched too. And the guarantees on TRN are impossible to quantify "without being able to characterize the rest of the traffic on that network."
>I.e., token passing strategies inherently implement a concept of >fairness and equal access. You have to *hope* your ethernet switch >does -- and try to get the vendor to provide you with quantitative >details of its (current!) implementation. Then, pray they never >discontinue it! *And* hope the rest of the nodes on YOUR network >behave equitably. > >[Getting details of switch internals is sort of like asking The Colonel >for his "secret recipe" [1]]
Plenty of switches implement enough VLAN and traffic shaping support to deal with almost all applications' requirements. TRN might have nominally better fairness (OK, you'll *definitely* get a chance to send a packet in the next several seconds some time), but you could still see packet losses and that did nothing for a receiver being too busy to pick up a packet. So you basically still have all the same problems to deal with. I'm not saying switched Ethernet solves all (or even any) of the problems, just that TRN really didn't either.
>I've been through this exercise before. Getting deterministic behavior >from ethernet -- WITHOUT PROTOCOL MODIFICATIONS -- is a real hassle. >Witness the assortment of automation protocols "over ethernet"... > >> *4472 bytes for 4Mb TRN, 17800 for 16Mb > >[1] Actually, I think an analysis of it was done in The Straight Dope >or one of Poundstone's books... short answer: "nothing special"
On Tue, 14 Jan 2014 12:44:38 -0600, Les Cargill
<lcargill99@comcast.com> wrote:

>Don Y wrote: >> Hi Robert, >> >> On 1/13/2014 9:55 PM, Robert Wessel wrote: >> >> [attrs elided] >> >>>>> The problem is you have to layer a *different* protocol onto those >>>>> media if you want deterministic behavior. AND, prevent any >>>>> "noncompliant" traffic from using the medium at the same time. >>>>> >>>>> E.g., you could have "office equipment" and "process control >>>>> equipment" sharing a token-passing network and *still* have >>>>> guarantees for the process control subsystems. Not the case >>>>> with things like ethernet (unless you create a special >>>>> protocol stack for those devices and/or interpose some bit of >>>>> kit that forces them to "behave" properly). >>>> >>>> As you are no doubt aware, what happened there was Ethernet >>>> switching and it well and truly solved the collision problem. >>>> >>>> Most, if not all packets live on a collision domain with exactly >>>> two NICs on it - except for 802.11>x< , where just about >>>> any concept smuggled form the other old standards doubtless lives in >>>> the air link. >>> >>> And given that most 100Mb and faster (wired) Ethernet links are full >>> duplex these days, there's effectively no collision domain at all. >>> >>> OTOH, that doesn't prevent the switch from dropping packets if the >>> destination port is sufficiently busy. >> >> Or for the actions of one node influencing the delivery of traffic >> from *another* node! ("Betty in accounting is printing a lengthy >> report -- the CNC machines have stopped as their input buffers are >> now empty...") > > >So VLANs... and 802.11Q or other CoS/QoS ....
As strange as it may sound, Ethernet is used on Airbus A350 and A380 planes. Of course the devices have strict throughput control mechanisms in the form of the AFDX protocol http://en.wikipedia.org/wiki/Avionics_Full-Duplex_Switched_Ethernet
Don Y wrote:
> Hi Les, > > On 1/13/2014 9:40 PM, Les Cargill wrote: >> Don Y wrote: > >>>>>>>> reliable ... but that got fixed and TR's predictable timing made >>>>>>>> analyzing systems and programming reliably timed delivery - >>>>>>>> particularly across repeaters - easier even than on CAN. >>>>>>> >>>>>>> At one time, I did an analysis that suggested even 4Mb TR would >>>>>>> outperform 10Mb ethernet when you were concerned with temporal >>>>>>> guarantees. >>>>>> >>>>>> Which was one of the touted features of TRN. Unfortunately for TRN, >>>>>> approximately zero users actually cared about that. >>>>> >>>>> It's too bad that "fast" has won out over "predictable" (in >>>>> many things -- not just network technology). >>>> >>>> Token rings, e.g. FDDI and others, had config/management >>>> problems that largely negated predictability guarantees, >>>> e.g. dropped tokens, duplicated tokens, complexity (have >>>> a look at all the FSMs!) >>>> >>>> CSMA/CD is much easier to manage and faultfind. >>> >>> The problem is you have to layer a *different* protocol onto those >>> media if you want deterministic behavior. AND, prevent any >>> "noncompliant" traffic from using the medium at the same time. >>> >>> E.g., you could have "office equipment" and "process control >>> equipment" sharing a token-passing network and *still* have >>> guarantees for the process control subsystems. Not the case >>> with things like ethernet (unless you create a special >>> protocol stack for those devices and/or interpose some bit of >>> kit that forces them to "behave" properly). >> >> As you are no doubt aware, what happened there was Ethernet >> switching and it well and truly solved the collision problem. > > It's not collisions that are the problem. Rather, it is > timeliness guarantees. A node on an ethernet switch has > no guarantee as to when -- or *if* -- its packets will > be delivered. >
It all depends on your application. You pay for better service.
> Switches have fixed size memories. There are no guarantees that > packets sent to the switch ever *go* anywhere. >
Sure. So don't run them at high utilization unless you have to. If you have to, get your checkbook out.
> By contrast, token passing networks gave assigned timeslots.
NO, they do not. TDM has timeslots; token passing { ARCnet, Token Ring } work differently for different switch topologies. Classic coax* Token Ring is a ring, and each NIC forwards on behalf of its neighbor unless the destination address is the NIC's address. *may also have been true of twisted pair; don't recall; most twisted pair ran just like Ethernet w.r.t cabling; the switches/hubs did all the footwork. Latter day Token Ring switches look just like Ethernet switches. Indeed, products would allow layer 2 switching between Ethernet and Token ring. At the very least they'd route. TDM is *a way*, but it's not *THE* way. If you can deal with retransmission, then Ethernet provides a lot from bandwidth for a lot less money.
> You *knew* when your "turn" to use the media would come along. > And, were *guaranteed* this by the basic design of the network > itself (not some other protocol layered on top of it). >
that "guarantee" is a holdover from the cognitive dissonance from the largely now-abandoned COTs network. I can sympathize with the desire to run a clock from Maine to San Diego, but .... what that cognitive dissonance was grounded in was the sure and certain knowledge that telecomms wasn't quite a market product, and some sort of central dogma was needed... It was all fine when a T1 was all the backhaul you'd ever need.
> Ever seen any timing guarantees for a generic network switch? > It's left as an exercise for the user: look at the packet buffer > size in the switch, determine whether it is a store-and-forward > switch
OOf. That gets ugly.
> or whether it can exploit cut-through technology, whether > it is blocking/nonblocking,
They're *all" blocking past some limit. > *and* the datacomm characteristics
> of all the other nodes on your network (will they cooperate > with each other's needs? Or, blindly use as much as they can > get??) and then try to come up with a hard-and-fast number > to determine the expected latency for a specific packet on a > specific node. > > Repeat the exercise for a token passing network. >
So just how low of a latency do you *need*? By "need", I mean "will negatively affect performance by this cost measure that relates to dollars." SO go gitcha a big ole ATM switch, and do that if it turns out that way. Maybe MPLS, other stuff. You will run into much larger timers in an IP stack than in the media, anyway. Meanwhile, *one* half-duplex RS485 link and...
> [If you think that ethernet makes those guarantees, then you > can elide all the acknowledgements in the protocols and still > be VERY CONFIDENT that everything STILL works properly :> ] >
But bit errors... Even RS232 over six inches will have a potential BER of more than (1/10^9th).
>> Most, if not all packets live on a collision domain with exactly >> two NICs on it - except for 802.11>x< , where just about >> any concept smuggled form the other old standards doubtless lives in >> the air link. >
-- Les Cargill
On Tue, 14 Jan 2014 13:04:24 -0600, Les Cargill
<lcargill99@comcast.com> wrote:


>TDM is *a way*, but it's not *THE* way. If you >can deal with retransmission, then Ethernet provides a lot >from bandwidth for a lot less money.
I was once looking for using 68360 and PPC QUICC communication processor TSA for TDMA multiplexing a low number of bits from a large number of nodes, but it did not materialize. One interesting alternative using at least some Ethernet hardware is the Ethernet Powerlink http://en.wikipedia.org/wiki/Ethernet_Powerlink which also can efficiently transfer a few bits from each mode.