I often use newlib standard C libraries with gcc toolchain for Cortex-M
platforms. It sometimes happens I need to manage calendar time: seconds
from 1970 or broken down time. And it sometimes happens I need to manage
timezone too, because the time reference comes from NTP (that is UTC).
newlib as expected defines a time() function that calls a syscall
function _gettimeofday(). It should be defined as in [1].
What is it? There's an assembler instruction that I don't understand:
asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0");
What is the cleanest way to override this behaviour and let newlib
time() to return a custom calendar time, maybe counted by a local RTC,
synchronized with a NTP?
The solution that comes to my mind is to override _gettimeofday() by
defining a custom function.
[1] https://github.com/eblot/newlib/blob/master/libgloss/arm/syscalls.c
newlib and time()
Started by ●September 30, 2022
Reply by ●September 30, 20222022-09-30
On 30/09/2022 08:29, pozz wrote:> I often use newlib standard C libraries with gcc toolchain for Cortex-M > platforms. It sometimes happens I need to manage calendar time: seconds > from 1970 or broken down time. And it sometimes happens I need to manage > timezone too, because the time reference comes from NTP (that is UTC). > > newlib as expected defines a time() function that calls a syscall > function _gettimeofday(). It should be defined as in [1]. > > What is it? There's an assembler instruction that I don't understand: > > asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); >It is a "software interrupt" instruction. If you have a separation of user-space and supervisor-space code in your system, this is the way you make a call to supervisor mode.> What is the cleanest way to override this behaviour and let newlib > time() to return a custom calendar time, maybe counted by a local RTC, > synchronized with a NTP? > > The solution that comes to my mind is to override _gettimeofday() by > defining a custom function. >Yes, that's the way to do it. Or define your own time functions that are appropriate to the task. I have almost never had a use for the standard library time functions - they are too much for most embedded systems which rarely need all the locale stuff, time zones, and tracking leap seconds, while lacking the stuff you /do/ need like high precision time counts. Use a single 64-bit monotonic timebase running at high speed (if your microcontroller doesn't support that directly, use a timer with an interrupt for tracking the higher part). That's enough for nanosecond precision for about 600 years. For human-friendly time and dates, either update every second or write your own simple second-to-human converter. It's easier if you have your base point relatively recently (there's no need to calculate back to 01.01.1970). If you have an internet connection, NTP is pretty simple if you are happy to use the NTP pools as a rough reference without trying to do millisecond synchronisation.> > > [1] https://github.com/eblot/newlib/blob/master/libgloss/arm/syscalls.c >
Reply by ●September 30, 20222022-09-30
On 30/9/22 16:29, pozz wrote:> I often use newlib standard C libraries with gcc toolchain for Cortex-M > platforms. It sometimes happens I need to manage calendar time: seconds > from 1970 or broken down time. And it sometimes happens I need to manage > timezone too, because the time reference comes from NTP (that is UTC). > > newlib as expected defines a time() function that calls a syscall > function _gettimeofday(). It should be defined as in [1]. > What is it? There's an assembler instruction that I don't understand:It's the internal implementation of a bog-standard BSD Unix system call. Have you tried `man 2 gettimeofday`? The first parameter points to a struct timeval with time_t tv_sec and suseconds_t tv_usec, and the second one (both optional) to a struct timezone with two ints called tz_minuteswest and tz_dsttime.> asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); > What is the cleanest way to override this behaviour and let newlib > time() to return a custom calendar time, maybe counted by a local RTC, > synchronized with a NTP?That depends on details of newlib and your tool chain. Clifford Heath
Reply by ●September 30, 20222022-09-30
Il 30/09/2022 09:04, David Brown ha scritto:> On 30/09/2022 08:29, pozz wrote: >> I often use newlib standard C libraries with gcc toolchain for >> Cortex-M platforms. It sometimes happens I need to manage calendar >> time: seconds from 1970 or broken down time. And it sometimes happens >> I need to manage timezone too, because the time reference comes from >> NTP (that is UTC). >> >> newlib as expected defines a time() function that calls a syscall >> function _gettimeofday(). It should be defined as in [1]. >> >> What is it? There's an assembler instruction that I don't understand: >> >> asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); >> > > It is a "software interrupt" instruction. If you have a separation of > user-space and supervisor-space code in your system, this is the way you > make a call to supervisor mode.Ok, but how that instruction helps in returning a value from _gettimeofday()?>> What is the cleanest way to override this behaviour and let newlib >> time() to return a custom calendar time, maybe counted by a local RTC, >> synchronized with a NTP? >> >> The solution that comes to my mind is to override _gettimeofday() by >> defining a custom function. >> > > Yes, that's the way to do it. > > Or define your own time functions that are appropriate to the task. I > have almost never had a use for the standard library time functions - > they are too much for most embedded systems which rarely need all the > locale stuff, time zones, and tracking leap seconds, while lacking the > stuff you /do/ need like high precision time counts. > > Use a single 64-bit monotonic timebase running at high speed (if your > microcontroller doesn't support that directly, use a timer with an > interrupt for tracking the higher part). That's enough for nanosecond > precision for about 600 years. > > For human-friendly time and dates, either update every second or write > your own simple second-to-human converter. It's easier if you have > your base point relatively recently (there's no need to calculate back > to 01.01.1970). > > If you have an internet connection, NTP is pretty simple if you are > happy to use the NTP pools as a rough reference without trying to do > millisecond synchronisation.I agree with you and I used to implement my own functions to manage calendar times. Sometimes I used internal or external RTC that gives date and time in broken down fields (seconds, minutes, ...). However most RTCs don't manage automatic DST (daylight saving time), so I started using a different approach. I started using a simple 32-bits timer that increments every 1 second. Maybe a timer clocked from an accurate 32.768kHz quartz with a 32768 prescaler (many RTC can be configured as a simple 32-bit counter). Rarely I need calendar times with a resolution better than 1 second. Now the big question: what the counter exactly represents? Of course, seconds elapsed from an epoch (that could be Unix 1970 or 2000 or 2020 or what you choose). But the real question is: UTC or localtime? I started using localtime, for example the timer counts seconds since year 2020 (so avoiding wrap-around at year 2038) in Rome timezone. However this approach pulls-in other issues. How to convert this number (seconds since 2020 in Rome) to broken-down time (day, month, hours...)? It's very complex, because you should count for leap years, but mostly for DST rules. In Rome we have a calendar times that occur two times, when the clock is moved backward by one hour for DST. What is the counter value of this times as seconds from epoch in Rome for this time? It's much more simple to start from seconds in UTC, as Linux (and maybe Windows) does. In this way you can use standard functions to convert seconds in UTC to localtime. For example, you can use localtime() (or localtime_r() that is better). Another bonus is when you have NTP, that returns seconds in UTC, so you can set your counter with the exact number retrived by NTP.
Reply by ●September 30, 20222022-09-30
On 30/9/22 19:29, pozz wrote:> Il 30/09/2022 09:04, David Brown ha scritto: >> On 30/09/2022 08:29, pozz wrote: >>> I often use newlib standard C libraries with gcc toolchain for >>> Cortex-M platforms. It sometimes happens I need to manage calendar >>> time: seconds from 1970 or broken down time. And it sometimes happens >>> I need to manage timezone too, because the time reference comes from >>> NTP (that is UTC). >>> >>> newlib as expected defines a time() function that calls a syscall >>> function _gettimeofday(). It should be defined as in [1]. >>> >>> What is it? There's an assembler instruction that I don't understand: >>> >>> asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); >> >> It is a "software interrupt" instruction. If you have a separation of >> user-space and supervisor-space code in your system, this is the way >> you make a call to supervisor mode. > > Ok, but how that instruction helps in returning a value from > _gettimeofday()?It traps into Kernel mode, with a different stack. The kernel uses memory manipulation to push return values into user-mode registers or the user stack as needed to simulate a procedure return.> It's much more simple to start from seconds in UTC, as Linux (and maybe > Windows) does. In this way you can use standard functions to convert > seconds in UTC to localtimeThat's a good way to always get the wrong result. You are ignoring the need for leap seconds. If you want a monotonic counter of seconds since some epoch, you must not use UTC, but TAI: <https://en.wikipedia.org/wiki/International_Atomic_Time> When I implmented this, I used a 64-bit counter in 100's of nanoseconds since a date about 6000BC, measuring in TAI. You can convert to UTC easily enough, and then use the timezone tables to get local times. Clifford Heath.
Reply by ●September 30, 20222022-09-30
On 30/09/2022 11:29, pozz wrote:> Il 30/09/2022 09:04, David Brown ha scritto: >> On 30/09/2022 08:29, pozz wrote: >>> I often use newlib standard C libraries with gcc toolchain for >>> Cortex-M platforms. It sometimes happens I need to manage calendar >>> time: seconds from 1970 or broken down time. And it sometimes happens >>> I need to manage timezone too, because the time reference comes from >>> NTP (that is UTC). >>> >>> newlib as expected defines a time() function that calls a syscall >>> function _gettimeofday(). It should be defined as in [1]. >>> >>> What is it? There's an assembler instruction that I don't understand: >>> >>> asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); >>> >> >> It is a "software interrupt" instruction. If you have a separation of >> user-space and supervisor-space code in your system, this is the way >> you make a call to supervisor mode. > > Ok, but how that instruction helps in returning a value from > _gettimeofday()?It will work if you have an OS that provides services to user-level code. The service type is passed in the SWI instruction (in this case, "SWI_Time"), and the service should return a value in r0. Calls like this are part of the "hosted" C library functions - they rely on a host OS to do the actual work.> > >>> What is the cleanest way to override this behaviour and let newlib >>> time() to return a custom calendar time, maybe counted by a local >>> RTC, synchronized with a NTP? >>> >>> The solution that comes to my mind is to override _gettimeofday() by >>> defining a custom function. >>> >> >> Yes, that's the way to do it. >> >> Or define your own time functions that are appropriate to the task. I >> have almost never had a use for the standard library time functions - >> they are too much for most embedded systems which rarely need all the >> locale stuff, time zones, and tracking leap seconds, while lacking the >> stuff you /do/ need like high precision time counts. >> >> Use a single 64-bit monotonic timebase running at high speed (if your >> microcontroller doesn't support that directly, use a timer with an >> interrupt for tracking the higher part). That's enough for nanosecond >> precision for about 600 years. >> >> For human-friendly time and dates, either update every second or write >> your own simple second-to-human converter. It's easier if you have >> your base point relatively recently (there's no need to calculate back >> to 01.01.1970). >> >> If you have an internet connection, NTP is pretty simple if you are >> happy to use the NTP pools as a rough reference without trying to do >> millisecond synchronisation. > > I agree with you and I used to implement my own functions to manage > calendar times. Sometimes I used internal or external RTC that gives > date and time in broken down fields (seconds, minutes, ...). > > However most RTCs don't manage automatic DST (daylight saving time), so > I started using a different approach. > I started using a simple 32-bits timer that increments every 1 second. > Maybe a timer clocked from an accurate 32.768kHz quartz with a 32768 > prescaler (many RTC can be configured as a simple 32-bit counter). > Rarely I need calendar times with a resolution better than 1 second. > > Now the big question: what the counter exactly represents? Of course, > seconds elapsed from an epoch (that could be Unix 1970 or 2000 or 2020 > or what you choose). But the real question is: UTC or localtime? > > I started using localtime, for example the timer counts seconds since > year 2020 (so avoiding wrap-around at year 2038) in Rome timezone. > However this approach pulls-in other issues. > > How to convert this number (seconds since 2020 in Rome) to broken-down > time (day, month, hours...)? It's very complex, because you should count > for leap years, but mostly for DST rules. > In Rome we have a calendar times that occur two times, when the clock is > moved backward by one hour for DST. What is the counter value of this > times as seconds from epoch in Rome for this time? > > It's much more simple to start from seconds in UTC, as Linux (and maybe > Windows) does. In this way you can use standard functions to convert > seconds in UTC to localtime. For example, you can use localtime() (or > localtime_r() that is better). > > Another bonus is when you have NTP, that returns seconds in UTC, so you > can set your counter with the exact number retrived by NTP. >That should all be fine.
Reply by ●September 30, 20222022-09-30
pozz <pozzugno@gmail.com> wrote:> I often use newlib standard C libraries with gcc toolchain for Cortex-M > platforms. It sometimes happens I need to manage calendar time: seconds > from 1970 or broken down time. And it sometimes happens I need to manage > timezone too, because the time reference comes from NTP (that is UTC). > > newlib as expected defines a time() function that calls a syscall > function _gettimeofday(). It should be defined as in [1]. > > What is it? There's an assembler instruction that I don't understand: > > asm ("swi %a1; mov %0, r0" : "=r" (value): "i" (SWI_Time) : "r0"); > > What is the cleanest way to override this behaviour and let newlib > time() to return a custom calendar time, maybe counted by a local RTC, > synchronized with a NTP?It appears 'libgloss' is the system-dependent part of newlib. It has various ideas of what those low level system functions should do, in particular linux-syscalls0.S, redboot-syscalls.c and syscalls.c (which appears to be calling Arm's Angel monitor). There's a guide for porting newlib to a new platform that describes what libgloss is and how to port it: https://sourceware.org/newlib/libgloss.html as well as: https://www.embecosm.com/appnotes/ean9/ean9-howto-newlib-1.0.html https://wiki.osdev.org/Porting_Newlib So the cleanest way wouldn't be to override _gettimeofday() as such, you'd make your own libgloss library that implemented the backend functions you wanted. Theo
Reply by ●September 30, 20222022-09-30
Il 30/09/2022 13:51, Clifford Heath ha scritto:> On 30/9/22 19:29, pozz wrote: >> Il 30/09/2022 09:04, David Brown ha scritto: >>> On 30/09/2022 08:29, pozz wrote:[...]>> It's much more simple to start from seconds in UTC, as Linux (and >> maybe Windows) does. In this way you can use standard functions to >> convert seconds in UTC to localtime > > That's a good way to always get the wrong result. You are ignoring the > need for leap seconds. If you want a monotonic counter of seconds since > some epoch, you must not use UTC, but TAI: > > <https://en.wikipedia.org/wiki/International_Atomic_Time> > > When I implmented this, I used a 64-bit counter in 100's of nanoseconds > since a date about 6000BC, measuring in TAI. You can convert to UTC > easily enough, and then use the timezone tables to get local times.What happens if the counter is UTC instead of IAT in a typical embedded application? There's a time when the counter is synchronized (by a manual operation from the user, by NTP or other means). At that time the broken-down time shown on the display is precise. You should wait for the next leap second to have an error of... 1 second.
Reply by ●September 30, 20222022-09-30
On 9/30/22 9:32 AM, pozz wrote:> Il 30/09/2022 13:51, Clifford Heath ha scritto: >> On 30/9/22 19:29, pozz wrote: >>> Il 30/09/2022 09:04, David Brown ha scritto: >>>> On 30/09/2022 08:29, pozz wrote: > > [...] > >>> It's much more simple to start from seconds in UTC, as Linux (and >>> maybe Windows) does. In this way you can use standard functions to >>> convert seconds in UTC to localtime >> >> That's a good way to always get the wrong result. You are ignoring the >> need for leap seconds. If you want a monotonic counter of seconds >> since some epoch, you must not use UTC, but TAI: >> >> <https://en.wikipedia.org/wiki/International_Atomic_Time> >> >> When I implmented this, I used a 64-bit counter in 100's of >> nanoseconds since a date about 6000BC, measuring in TAI. You can >> convert to UTC easily enough, and then use the timezone tables to get >> local times. > > What happens if the counter is UTC instead of IAT in a typical embedded > application? There's a time when the counter is synchronized (by a > manual operation from the user, by NTP or other means). At that time the > broken-down time shown on the display is precise. > > You should wait for the next leap second to have an error of... 1 second. > >The key thing to remember is there is more than one system of time. As I remember time() returns UTC seconds since the epoch, which ignores leap seconds. This means the time() function will either pause for 1 second during the leap second, or some systems smear the 1 second anomoly over a period of time. (This smeared UTC is monotonic, if not exactly accurate during the smear period). For time() ALL days are 24*60*60 seconds long. There are other time systems (Like the TAI) that keep track of leap seconds, but then to use those to convert to wall-clock time, you need a historical table of when leap seconds occured, and you need to either refuse to handle the farther future or admit you are needing to "Guess" when those leap seconds will need to be applied. Most uses of TAI time are for just short intervals without a need to convert to wall clock.
Reply by ●September 30, 20222022-09-30
On 9/30/2022 2:29 AM, pozz wrote:> Another bonus is when you have NTP, that returns seconds in UTC, so you can set > your counter with the exact number retrived by NTP.No. You always have to ensure that time keeps flowing in one direction. So, time either "doesn't exist" before your initial sync with the time server (what if the server isn't available when you want to do that?) *or* you have to look at your current notion of "now" and ensure that the "real" value of now, when obtained from the time server, is always in the future relative to your notion. [Note that NTP slaves don't blindly assume the current time is as reported but *slew* to the new value, over some interval.] This also ignores the possibility of computations with relative *intervals* being inconsistent with these spontaneous "resets".
