EmbeddedRelated.com
Forums
The 2024 Embedded Online Conference

clock synchronization over IP

Started by Unknown August 24, 2018
On Tuesday, September 18, 2018 at 2:11:06 PM UTC+12, George Neuner wrote:
> On Mon, 17 Sep 2018 15:04:58 -0700 (PDT), gp....@gmail.com wrote: > > >On Tuesday, September 18, 2018 at 3:30:50 AM UTC+12, George Neuner wrote: > >> > >> I think you said you needed +-20ms ... you didn't say how large or > >> complex the network, but if the cross section is under 10 milliseconds > >> (a pretty large wired net), you should be easily able to achieve your > >> desired resolution just with software PTP. > > > >Thanks. It's for synching audio so +-20ms is ok. There can be up to > >64 nodes. We can assume a dedicated network but we may have some > >VOIP traffic with up to six people on a phone conference call. We > >were intending to use NTP in hierarchical mode because it seems > >easier to configure. We need the synch to keep working if there's a > >break in the network or a node is offline. > > Are these nodes some kind of networkable speakers? > > +-20ms is no good for audio. Ears are a lot more sensitive to > syncronization artifacts than are eyes. Even "tin" ears will perceive > some artifacts at +-20ms. Depending on content [e.g., people tolerate > hiccups in voice better than in music] stereo streams need to be > sync'd within +-8ms to be acceptible to most people. Under optimal > listening conditions, audiophiles with really good ears can hear sync > artifacts even down to +-3ms. > > That said, 64 nodes is not all that many - cable lengths permitting > they all could be on one switch. Even with a handful of switches, the > cross section of the LAN should be < ~3ms. Unless you have a > congested network or serious drift problems with your nodes, NTP can > be about as accurate as the network cross section. > > Software PTP can do better than NTP - if you can afford the protocol > traffic it can get down to the accuracy of the timekeeper clock. But > if you want reliable sub-millisecond synchronization with minimum > traffic then you are going to need PTP hardware. > > YMMV, > George
Yes, it's a network of speakers. It sounds like NTP should be ok then. We're likely to use a hierarchical arrangement. Do you know if there's any way to use either PTP or NTP without configuring lots of static IP addresses at every node.
On 2018-09-18, graeme.prentice@gmail.com <graeme.prentice@gmail.com> wrote:
> > Yes, it's a network of speakers. It sounds like NTP should be ok then. We're likely to use a hierarchical arrangement. Do you know if there's any way to use either PTP or NTP without configuring lots of static IP addresses at every node.
You should only need one, or a handful for redundancy. It's been a while since I configured NTP from scratch but seem to remember there are a couple of ways it can be configured - either peer to peer where it averages out clock differences or client/server where the clients follow the server clock. In the latter case only the server(s) need a static IP address. That is essentially the way I have configured it in the past - clients follow the server which itself gets the time from the Internet. I figure that way the local clocks are synced with each relatively well (without the vagaries of the Internet link) and are in reasonable sync with the real world time. -- Andrew Smallshaw andrews@sdf.org
graeme.prentice@gmail.com writes:

> Yes, it's a network of speakers. It sounds like NTP should be ok > then. We're likely to use a hierarchical arrangement. Do you know if > there's any way to use either PTP or NTP without configuring lots of > static IP addresses at every node.
If you're running DHCP, use DHCP option 42 to send addresses of NTP servers to the nodes. -- mikko
On 08/25/18 01:54, gp.kiwi@gmail.com wrote:
> We need to synchronize clocks (so we can synchronize audio) on physically separated controllers that are connected via ethernet that may or may not be connected to the internet. The controllers have ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more than 50 ms difference but preferably less. Synchronization needs to continue working between any two controllers regardless of the failure of any other controller. > > Does anyone have an idea what is the easiest way to do this?
Read all the replies about ntp etc, but perhaps look at this another way ?. Rather than each client *requesting* time, how about a server / node broadcasting time to the net, with each client locking on to that ?. Not sure if there are any standards for it, but may be be worth having a look. That would get rid of any client / server request delays altogether, with sync offsets determined entirely by the client software, probably down to microseconds... Chris
On 21.9.2018 &#1075;. 17:00, Chris wrote:
> On 08/25/18 01:54, gp.kiwi@gmail.com wrote: >> We need to synchronize clocks (so we can synchronize audio) on >> physically separated controllers that are connected via ethernet that >> may or may not be connected to the internet. The controllers have >> ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more >> than 50 ms difference but preferably less. Synchronization needs to >> continue working between any two controllers regardless of the failure >> of any other controller. >> >> Does anyone have an idea what is the easiest way to do this? > > Read all the replies about ntp etc, but perhaps > look at this another way ?. Rather than each client > *requesting* time, how about a server / node > broadcasting time to the net, with each client > locking on to that ?. Not sure if there are any > standards for it, but may be be worth having > a look. > > That would get rid of any client / server request > delays altogether, with sync offsets determined > entirely by the client software, probably down > to microseconds... > > Chris >
I think the idea is excellent. Broadcast a packet - on the same Ethernet branch (as is the case of the OP I think) there will be no skew to talk about; then have all recipients confirm to the server they got the time, and once the server receives all confirmations from all active hosts (it will need to know which these are) it safely (via tcp) tells each host the broadcast time it got is now universally valid. Dimiter ====================================================== Dimiter Popoff, TGI http://www.tgi-sci.com ====================================================== http://www.flickr.com/photos/didi_tgi/
On Fri, 21 Sep 2018 17:12:00 +0300, Dimiter_Popoff <dp@tgi-sci.com>
wrote:

>On 21.9.2018 ?. 17:00, Chris wrote: >> On 08/25/18 01:54, gp.kiwi@gmail.com wrote: >>> We need to synchronize clocks (so we can synchronize audio) on >>> physically separated controllers that are connected via ethernet that >>> may or may not be connected to the internet. The controllers have >>> ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more >>> than 50 ms difference but preferably less. Synchronization needs to >>> continue working between any two controllers regardless of the failure >>> of any other controller. >>> >>> Does anyone have an idea what is the easiest way to do this? >> >> Read all the replies about ntp etc, but perhaps >> look at this another way ?. Rather than each client >> *requesting* time, how about a server / node >> broadcasting time to the net, with each client >> locking on to that ?. Not sure if there are any >> standards for it, but may be be worth having >> a look. >> >> That would get rid of any client / server request >> delays altogether, with sync offsets determined >> entirely by the client software, probably down >> to microseconds... >> >> Chris >> > >I think the idea is excellent. Broadcast a packet - on the same >Ethernet branch (as is the case of the OP I think) there will be >no skew to talk about; then have all recipients confirm to the >server they got the time, and once the server receives all confirmations >from all active hosts (it will need to know which these are) it safely >(via tcp) tells each host the broadcast time it got is now universally >valid.
Unless you're using one of the shared media versions of Ethernet, which is somewhat unlikely these days, although not impossible, or if your "switch" is really a hub (aka a repeater), also unlikely, except in really inexpensive gear, there's no guarantee that the switch isn't going to introduce variable latency in forwarding broadcast frames to the various ports. In fact the switch will have to introduce skew if on of the ports has queued transmission traffic, and the others don't, or if the ports are not all running at the same speed. Slew can be introduced at the receiving nodes as well based on how likely they have any queued receive traffic, either in the NIC or in OS buffers. A question would be how close to the wire the received frame can be timestamped. That might also leave all the nodes, except the time server, synchronized, unless the time server can also receive the broadcast, which is not a universal capability (the sending node often cannot). That's not an issue if the time server node doesn't need to be synchronized with the other nodes, but I'd generally assume that being the time server was just a function of one of the nodes doing "normal" work. So there's a lot to be said for an interactive approach.
On 09/21/18 15:12, Dimiter_Popoff wrote:
> On 21.9.2018 &#1075;. 17:00, Chris wrote: >> On 08/25/18 01:54, gp.kiwi@gmail.com wrote: >>> We need to synchronize clocks (so we can synchronize audio) on >>> physically separated controllers that are connected via ethernet that >>> may or may not be connected to the internet. The controllers have >>> ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more >>> than 50 ms difference but preferably less. Synchronization needs to >>> continue working between any two controllers regardless of the failure >>> of any other controller. >>> >>> Does anyone have an idea what is the easiest way to do this? >> >> Read all the replies about ntp etc, but perhaps >> look at this another way ?. Rather than each client >> *requesting* time, how about a server / node >> broadcasting time to the net, with each client >> locking on to that ?. Not sure if there are any >> standards for it, but may be be worth having >> a look. >> >> That would get rid of any client / server request >> delays altogether, with sync offsets determined >> entirely by the client software, probably down >> to microseconds... >> >> Chris >> > > I think the idea is excellent. Broadcast a packet - on the same > Ethernet branch (as is the case of the OP I think) there will be > no skew to talk about; then have all recipients confirm to the > server they got the time, and once the server receives all confirmations > from all active hosts (it will need to know which these are) it safely > (via tcp) tells each host the broadcast time it got is now universally > valid. > > Dimiter > > ====================================================== > Dimiter Popoff, TGI http://www.tgi-sci.com > ====================================================== > http://www.flickr.com/photos/didi_tgi/ > > >
If you wanted to use really low level, you might be able to (mis) use an existing protocol like a rarp b/cast, to get the job done, depending on network requirements. Had a quick search and there are other methods, some described here: http://tinyos.stanford.edu/tinyos-wiki/index.php/FTSP You can also buy gps rx engines / boards really cheaply now and most of those have a 1 pps o/p at ttl level, as well as time of day via serial port. Antenna could be the sticking point there though. Just seems to me that ntp is too high level, with not enough granularity for audio synchronisation... Chris
On 09/21/18 15:58, Robert Wessel wrote:
> On Fri, 21 Sep 2018 17:12:00 +0300, Dimiter_Popoff<dp@tgi-sci.com> > wrote: > >> On 21.9.2018 ?. 17:00, Chris wrote: >>> On 08/25/18 01:54, gp.kiwi@gmail.com wrote: >>>> We need to synchronize clocks (so we can synchronize audio) on >>>> physically separated controllers that are connected via ethernet that >>>> may or may not be connected to the internet. The controllers have >>>> ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more >>>> than 50 ms difference but preferably less. Synchronization needs to >>>> continue working between any two controllers regardless of the failure >>>> of any other controller. >>>> >>>> Does anyone have an idea what is the easiest way to do this? >>> >>> Read all the replies about ntp etc, but perhaps >>> look at this another way ?. Rather than each client >>> *requesting* time, how about a server / node >>> broadcasting time to the net, with each client >>> locking on to that ?. Not sure if there are any >>> standards for it, but may be be worth having >>> a look. >>> >>> That would get rid of any client / server request >>> delays altogether, with sync offsets determined >>> entirely by the client software, probably down >>> to microseconds... >>> >>> Chris >>> >> >> I think the idea is excellent. Broadcast a packet - on the same >> Ethernet branch (as is the case of the OP I think) there will be >> no skew to talk about; then have all recipients confirm to the >> server they got the time, and once the server receives all confirmations >>from all active hosts (it will need to know which these are) it safely >> (via tcp) tells each host the broadcast time it got is now universally >> valid. > > > Unless you're using one of the shared media versions of Ethernet, > which is somewhat unlikely these days, although not impossible, or if > your "switch" is really a hub (aka a repeater), also unlikely, except > in really inexpensive gear, there's no guarantee that the switch isn't > going to introduce variable latency in forwarding broadcast frames to > the various ports. In fact the switch will have to introduce skew if > on of the ports has queued transmission traffic, and the others don't, > or if the ports are not all running at the same speed. > > Slew can be introduced at the receiving nodes as well based on how > likely they have any queued receive traffic, either in the NIC or in > OS buffers. A question would be how close to the wire the received > frame can be timestamped. > > That might also leave all the nodes, except the time server, > synchronized, unless the time server can also receive the broadcast, > which is not a universal capability (the sending node often cannot). > That's not an issue if the time server node doesn't need to be > synchronized with the other nodes, but I'd generally assume that being > the time server was just a function of one of the nodes doing "normal" > work. > > So there's a lot to be said for an interactive approach.
Agreed, but the spec is to synchronise audio, which is a hard real time requirement. You might have to consider other issues as well, such as embedded Linux real time response time. That said, if you have control over the network topology, you can specify which switches and hubs are permissable and design around that. Lowest cost that meets the spec is usually the aim and the simplest protocol that gets the job done can also minimise latency and hardware requirements :-)... Chris
On Fri, 21 Sep 2018 15:00:51 +0100, Chris <xxx.syseng.yyy@gfsys.co.uk>
wrote:

>On 08/25/18 01:54, gp.kiwi@gmail.com wrote: >>We need to synchronize clocks (so we can synchronize audio) on >>physically separated controllers that are connected via ethernet that >>may or may not be connected to the internet. The controllers have >>ARM9 LPC3250 SOM and embedded Linux. We need to synch with no more >>than 50 ms difference but preferably less. Synchronization needs to >>continue working between any two controllers regardless of the >>failure of any other controller. >> >>Does anyone have an idea what is the easiest way to do this? > >Read all the replies about ntp etc, but perhaps >look at this another way ?. Rather than each client >*requesting* time, how about a server / node >broadcasting time to the net, with each client >locking on to that ?. Not sure if there are any >standards for it, but may be be worth having >a look. > >That would get rid of any client / server request >delays altogether, with sync offsets determined >entirely by the client software, probably down >to microseconds...
There will be slew along the any cable and through any repeater - so it is impossible to avoid altogether. On a common cable, nodes nearer the broadcast point receive it sooner than those further away. Twisted pair is a (multi)star configuration centered around a switch or hub which is has a repeater per port. Moreover, simply broadcasting the time from a master won't work[*] ... in order to synchronize the clocks youu need to know for each node (to a good approximation) the time it takes for the network to deliver the time packet. [*] except in a dedicated setup like a DGPS timing network. The problem with such setups is that all the connections from the master to the slaves need to be exactly the same length (bit delay). That is very hard to achieve in most installations. George
On 09/21/18 22:15, George Neuner wrote:

> Moreover, simply broadcasting the time from a master won't work[*] ... > in order to synchronize the clocks youu need to know for each node (to > a good approximation) the time it takes for the network to deliver the > time packet. > > [*] except in a dedicated setup like a DGPS timing network. The > problem with such setups is that all the connections from the master > to the slaves need to be exactly the same length (bit delay). That is > very hard to achieve in most installations. > > George
There is cable delay, but that's in the order a few nS per foot at most, where the requirement here is in mS, so that can probably be ignored. As for the switch or hub delay, that could be modelled or even subject to testing to find suitable products. Ideally, I would use gps to sync, as it has all the outputs required to within a few nS over a wide area, but you need a clear view of the sky to make that work. Still think it would be worth looking at broadcast methods, as it could be made low cost at client and server ends, could be run on modest h/w without use of an OS, even without a full tcp/ip stack and is probably good enough for the application... Chris

The 2024 Embedded Online Conference