clock synchronization over IP

We need to synchronize clocks (so we can synchronize audio) on physically separated controllers that are connected via ethernet that may or may not be connected to the internet.  The controllers have ARM9 LPC3250 SOM and embedded Linux.  We need to synch with no more than 50 ms difference but preferably less.  Synchronization needs to continue working between any two controllers regardless of the failure of any other controller.

Does anyone have an idea what is the easiest way to do this?  

gp.kiwi@gmail.com writes:
> than 50 ms difference but preferably less....
> Does anyone have an idea what is the easiest way to do this?  

Ordinary NTP should be able to do that, especially over local ethernet.

gp....@gmail.com wrote:
> We need to synchronize clocks (so we can synchronize audio) on physically separated controllers that are connected via ethernet that may or may not be connected to the internet.  The controllers have ARM9 LPC3250 SOM and embedded Linux.  We need to synch with no more than 50 ms difference but preferably less.  Synchronization needs to continue working between any two controllers regardless of the failure of any other controller.
> 
> Does anyone have an idea what is the easiest way to do this?

If you need better precision than NTP gives, you should take a look at PTP ( http://www.nist.gov/el/isd/ieee/ieee1588.cfm ).
For extremally high precision (sub ns) there is White Rabbit ( https://www.ohwr.org/projects/white-rabbit/wiki/synchronization ).

HTH & Regards,
Wojtek

On Sunday, September 16, 2018 at 4:00:57 AM UTC+12, wza...@gmail.com wrote:
> gp....@gmail.com wrote:
> > We need to synchronize clocks (so we can synchronize audio) on physically separated controllers that are connected via ethernet that may or may not be connected to the internet.  The controllers have ARM9 LPC3250 SOM and embedded Linux.  We need to synch with no more than 50 ms difference but preferably less.  Synchronization needs to continue working between any two controllers regardless of the failure of any other controller.
> > 
> > Does anyone have an idea what is the easiest way to do this?
> 
> If you need better precision than NTP gives, you should take a look at PTP ( http://www.nist.gov/el/isd/ieee/ieee1588.cfm ).
> For extremally high precision (sub ns) there is White Rabbit ( https://www.ohwr.org/projects/white-rabbit/wiki/synchronization ).
> 
> HTH & Regards,
> Wojtek

Thanks, I did eventually discover PTP.  The hardware timestamping appeals to me but PTP is apparently harder to set up and you need routers and switches that support it.  I found an article that says this.
<quote>
In the case of PTP a slave synchronizes to a master clock, which other masters listen in, so that they can take over if the active master goes away for any reason. That&rsquo;s good, but NTP does one better. The client gets time from all of the servers and ignore one which seems to be too far off in time compared to the other servers.  </>

https://blog.meinbergglobal.com/2013/11/22/ntp-vs-ptp-network-timing-smackdown/

I'm concerned that NTP is more for synchronizing "clock time" rather than audio but if it does what the above article says then it should be ok.

On Sat, 15 Sep 2018 15:08:11 -0700 (PDT), gp.kiwi@gmail.com wrote:


>Thanks, I did eventually discover PTP.  The hardware timestamping
>appeals to me but PTP is apparently harder to set up and you need
>routers and switches that support it. 

PTP doesn't need any special network hardware - it implemented over
UDP.

George

On Monday, September 17, 2018 at 12:49:25 AM UTC+12, George Neuner wrote:
> On Sat, 15 Sep 2018 15:08:11 -0700 (PDT), gp...@gmail.com wrote:
> 
> 
> >Thanks, I did eventually discover PTP.  The hardware timestamping
> >appeals to me but PTP is apparently harder to set up and you need
> >routers and switches that support it. 
> 
> PTP doesn't need any special network hardware - it implemented over
> UDP.
> 
> George

After googling for information it seems that some hardware does support hardware timestamping and there are implementations of both NTP and PTP that use it.
https://en.wikipedia.org/wiki/List_of_PTP_implementations

The main wikipedia article on PTP actually gives (almost) no indication that hardware timestamping gives better accuracy and claims that on a LAN, it can achieve sub-microsecond accuracy.  Without hardware support I suspect that kind of accuracy requires a low traffic LAN, although it's possible that it keeps trying until it achieves a really short round-trip time.

On Mon, 17 Sep 2018 01:45:53 -0700 (PDT), gp.kiwi@gmail.com wrote:

>On Monday, September 17, 2018 at 12:49:25 AM UTC+12, George Neuner wrote:
>> On Sat, 15 Sep 2018 15:08:11 -0700 (PDT), gp...@gmail.com wrote:
>> 
>> 
>> >Thanks, I did eventually discover PTP.  The hardware timestamping
>> >appeals to me but PTP is apparently harder to set up and you need
>> >routers and switches that support it. 
>> 
>> PTP doesn't need any special network hardware - it implemented over
>> UDP.
>> 
>> George
>
>After googling for information it seems that some hardware does
>support hardware timestamping and there are implementations of both
>NTP and PTP that use it.
>https://en.wikipedia.org/wiki/List_of_PTP_implementations
>
>The main wikipedia article on PTP actually gives (almost) no
>indication that hardware timestamping gives better accuracy and
>claims that on a LAN, it can achieve sub-microsecond accuracy. 
>Without hardware support I suspect that kind of accuracy requires a
>low traffic LAN, although it's possible that it keeps trying until it
>achieves a really short round-trip time.

You need enough protocol packets to get through (to every node) in a
timely manner to correct for the node's clock drift.  But what you
need depends on how good (or bad) is the clock.  You could need
several packets/second, or you might need just a few packets/minute,
depending ...

Obviously you need to start with high precision time source ... an
easy way to do it is to buy a network switch that has PTP built in.
But in a simple setup your time source could be just some node
designated to be the timekeeper.

You don't need routers/bridges to implement the protocol unless there
is significant congestion and lots of variance in packet time across
the network.  A few milliseconds here and there will be sorted out by
the protocol.

I think you said you needed +-20ms ... you didn't say how large or
complex the network, but if the cross section is under 10 milliseconds
(a pretty large wired net), you should be easily able to achieve your
desired resolution just with software PTP.

YMMV,
George

On Tuesday, September 18, 2018 at 3:30:50 AM UTC+12, George Neuner wrote:
> On Mon, 17 Sep 2018 01:45:53 -0700 (PDT), gp...@gmail.com wrote:
> 
> >On Monday, September 17, 2018 at 12:49:25 AM UTC+12, George Neuner wrote:
> >> On Sat, 15 Sep 2018 15:08:11 -0700 (PDT), gp...@gmail.com wrote:
> >> 
> >> 
> >> >Thanks, I did eventually discover PTP.  The hardware timestamping
> >> >appeals to me but PTP is apparently harder to set up and you need
> >> >routers and switches that support it. 
> >> 
> >> PTP doesn't need any special network hardware - it implemented over
> >> UDP.
> >> 
> >> George
> >
> >After googling for information it seems that some hardware does
> >support hardware timestamping and there are implementations of both
> >NTP and PTP that use it.
> >https://en.wikipedia.org/wiki/List_of_PTP_implementations
> >
> >The main wikipedia article on PTP actually gives (almost) no
> >indication that hardware timestamping gives better accuracy and
> >claims that on a LAN, it can achieve sub-microsecond accuracy. 
> >Without hardware support I suspect that kind of accuracy requires a
> >low traffic LAN, although it's possible that it keeps trying until it
> >achieves a really short round-trip time.
> 
> You need enough protocol packets to get through (to every node) in a
> timely manner to correct for the node's clock drift.  But what you
> need depends on how good (or bad) is the clock.  You could need
> several packets/second, or you might need just a few packets/minute,
> depending ...
> 
> Obviously you need to start with high precision time source ... an
> easy way to do it is to buy a network switch that has PTP built in.
> But in a simple setup your time source could be just some node
> designated to be the timekeeper.
> 
> You don't need routers/bridges to implement the protocol unless there
> is significant congestion and lots of variance in packet time across
> the network.  A few milliseconds here and there will be sorted out by
> the protocol.
> 
> I think you said you needed +-20ms ... you didn't say how large or
> complex the network, but if the cross section is under 10 milliseconds
> (a pretty large wired net), you should be easily able to achieve your
> desired resolution just with software PTP.
> 
> YMMV,
> George

Thanks.  It's for synching audio so +-20ms is ok.  There can be up to 64 nodes.  We can assume a dedicated network but we may have some VOIP traffic with up to six people on a phone conference call.  We were intending to use NTP in hierarchical mode because it seems easier to configure.  We need the synch to keep working if there's a break in the network or a node is offline.

On Mon, 17 Sep 2018 15:04:58 -0700 (PDT), gp.kiwi@gmail.com wrote:

>On Tuesday, September 18, 2018 at 3:30:50 AM UTC+12, George Neuner wrote:
>>
>> I think you said you needed +-20ms ... you didn't say how large or
>> complex the network, but if the cross section is under 10 milliseconds
>> (a pretty large wired net), you should be easily able to achieve your
>> desired resolution just with software PTP.
>
>Thanks.  It's for synching audio so +-20ms is ok.  There can be up to
>64 nodes.  We can assume a dedicated network but we may have some
>VOIP traffic with up to six people on a phone conference call.  We
>were intending to use NTP in hierarchical mode because it seems
>easier to configure.  We need the synch to keep working if there's a
>break in the network or a node is offline.

Are these nodes some kind of networkable speakers?

+-20ms is no good for audio.  Ears are a lot more sensitive to
syncronization artifacts than are eyes.  Even "tin" ears will perceive
some artifacts at +-20ms.  Depending on content [e.g., people tolerate
hiccups in voice better than in music] stereo streams need to be
sync'd within +-8ms to be acceptible to most people. Under optimal
listening conditions, audiophiles with really good ears can hear sync
artifacts even down to +-3ms.

That said, 64 nodes is not all that many - cable lengths permitting
they all could be on one switch.  Even with a handful of switches, the
cross section of the LAN should be < ~3ms.  Unless you have a
congested network or serious drift problems with your nodes, NTP can
be about as accurate as the network cross section.

Software PTP can do better than NTP - if you can afford the protocol
traffic it can get down to the accuracy of the timekeeper clock.  But
if you want reliable sub-millisecond synchronization with minimum
traffic then you are going to need PTP hardware.

YMMV,
George

On 17/09/18 18:45, gp.kiwi@gmail.com wrote:
> Without hardware support I suspect that kind of accuracy requires a low traffic LAN, although it's possible that it keeps trying until it achieves a really short round-trip time.


NTP tries to measure the minimum round-trip time, then halve
it to get the one-way time, and choose the packets that
arrived quickest. It's surprisingly good even on a busy
network, because it can detect that and spend longer
searching for "good" round trips.