On 6/23/2016 4:45 PM, Clifford Heath wrote:> On 23/06/16 23:05, Reinhardt Behm wrote: >> Jack wrote: >>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>> I'm going to develop a local/remote control of an electronic device. It >>>> communicates through a RS485 link. >>>> I'm thinking to use SQLite database to store the data. The poller writes >>>> the database, HTTP and QT reads from it. It seems SQLite will take care >>>> the multi-thread/multi-process scenario. > > SQLite "takes care of" that by using a single giant lock. > That might work for you, since it sounds like you probably > have very easy throughput and latency demands. It would > be a good solution if you weren't using Qt. Les Cargill > hasn't seen how remarkably lightweight SQLite actually is. > >> Since you already using Qt, I can propose a solution which I use quite >> often. It is even flying in more than 100 helicopters. Yes, penguins can fly >> The poller fetches that data from the remote device via RS485 and keeps a >> for it local copy. If permanent storage is a requirement it can also take >> care of this. >> In addition it open a QLocalServer which listens for connections from other >> processes. >> The other (multiple) processes connect to this server using a QLocalSocket. >> They can then poll are are sent the data directly when the poller receives >> new data. This way no locking is required since the poller does its sending >> in a single thread. > > Reinhardt, do you deal with write-blocking if the reader fails? > If the write is in the poller thread, any blocking on writes will > delay the polling - unless you use non-blocking writes. > >> This Server/Socket combination has the advantage that it easily integrates >> into the signal/slot mechanism of Qt. > > I think it's good advice, though I wouldn't personally have chosen Qt. > > This application is not demanding enough to need a shared memory > solution, with the attendant need for locks and especially, > memory barriers, of which most respondents seem ill-educated.That's only a problem if you "roll your own" locks. IME, the SysV shared memory operations also carry along the SysV semaphores. And, you *want* the OS to know about any locks you've implemented as, inevitably, you'll want to add another "client" down the road and then have to worry about the various types of deadlock that can occur (that the OS can mitigate, in some instances). E.g., the "single giant lock" that SQLite uses can similarly be implemented to control ALL accesses to the shared memory segment. You also have the advantage of *seeing* all of the code that is accessing these shared objects -- instead of wondering what SQLite is doing and when its doing it.> In particular, Mel Wilson's "one writer, many reader" is no > longer so simple with modern multi-core architectures. Writes > do not have to happen in the order they are written, so readers > can get confused.
Embedded Linux: share data among different processes
Started by ●June 23, 2016
Reply by ●June 23, 20162016-06-23
Reply by ●June 24, 20162016-06-24
On 24/06/16 10:19, Don Y wrote:> On 6/23/2016 4:45 PM, Clifford Heath wrote: >> On 23/06/16 23:05, Reinhardt Behm wrote: >>> Jack wrote: >>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>> I'm going to develop a local/remote control of an electronic >>>>> device. It >>>>> communicates through a RS485 link. >>>>> I'm thinking to use SQLite database to store the data. The poller >>>>> writes >>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>> care >>>>> the multi-thread/multi-process scenario. >> >> SQLite "takes care of" that by using a single giant lock. >> That might work for you, since it sounds like you probably >> have very easy throughput and latency demands. It would >> be a good solution if you weren't using Qt. Les Cargill >> hasn't seen how remarkably lightweight SQLite actually is. >> >>> Since you already using Qt, I can propose a solution which I use quite >>> often. It is even flying in more than 100 helicopters. Yes, penguins >>> can fly >>> The poller fetches that data from the remote device via RS485 and >>> keeps a >>> for it local copy. If permanent storage is a requirement it can also >>> take >>> care of this. >>> In addition it open a QLocalServer which listens for connections from >>> other >>> processes. >>> The other (multiple) processes connect to this server using a >>> QLocalSocket. >>> They can then poll are are sent the data directly when the poller >>> receives >>> new data. This way no locking is required since the poller does its >>> sending >>> in a single thread. >> >> Reinhardt, do you deal with write-blocking if the reader fails? >> If the write is in the poller thread, any blocking on writes will >> delay the polling - unless you use non-blocking writes. >> >>> This Server/Socket combination has the advantage that it easily >>> integrates >>> into the signal/slot mechanism of Qt. >> >> I think it's good advice, though I wouldn't personally have chosen Qt. >> >> This application is not demanding enough to need a shared memory >> solution, with the attendant need for locks and especially, >> memory barriers, of which most respondents seem ill-educated. > > That's only a problem if you "roll your own" locks. IME, the SysV > shared memory operations also carry along the SysV semaphores.As I said, ill-educated. As you'll find out when the SysV semaphore grants you access to some shared memory resource that has been written, but only some of the writes have been flushed. If you're on hardware that can do this, you need to know about it, or leave the work to someone else who does... like the authors of SQLite. Clifford Heath.> > And, you *want* the OS to know about any locks you've implemented > as, inevitably, you'll want to add another "client" down the road > and then have to worry about the various types of deadlock that > can occur (that the OS can mitigate, in some instances). > > E.g., the "single giant lock" that SQLite uses can similarly be > implemented to control ALL accesses to the shared memory segment. > > You also have the advantage of *seeing* all of the code that is accessing > these shared objects -- instead of wondering what SQLite is doing > and when its doing it. > >> In particular, Mel Wilson's "one writer, many reader" is no >> longer so simple with modern multi-core architectures. Writes >> do not have to happen in the order they are written, so readers >> can get confused. >
Reply by ●June 24, 20162016-06-24
On 6/23/2016 8:25 PM, Clifford Heath wrote:> On 24/06/16 10:19, Don Y wrote: >> On 6/23/2016 4:45 PM, Clifford Heath wrote: >>> On 23/06/16 23:05, Reinhardt Behm wrote: >>>> Jack wrote: >>>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>>> I'm going to develop a local/remote control of an electronic >>>>>> device. It >>>>>> communicates through a RS485 link. >>>>>> I'm thinking to use SQLite database to store the data. The poller >>>>>> writes >>>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>>> care >>>>>> the multi-thread/multi-process scenario. >>> >>> SQLite "takes care of" that by using a single giant lock. >>> That might work for you, since it sounds like you probably >>> have very easy throughput and latency demands. It would >>> be a good solution if you weren't using Qt. Les Cargill >>> hasn't seen how remarkably lightweight SQLite actually is. >>> >>>> Since you already using Qt, I can propose a solution which I use quite >>>> often. It is even flying in more than 100 helicopters. Yes, penguins >>>> can fly >>>> The poller fetches that data from the remote device via RS485 and >>>> keeps a >>>> for it local copy. If permanent storage is a requirement it can also >>>> take >>>> care of this. >>>> In addition it open a QLocalServer which listens for connections from >>>> other >>>> processes. >>>> The other (multiple) processes connect to this server using a >>>> QLocalSocket. >>>> They can then poll are are sent the data directly when the poller >>>> receives >>>> new data. This way no locking is required since the poller does its >>>> sending >>>> in a single thread. >>> >>> Reinhardt, do you deal with write-blocking if the reader fails? >>> If the write is in the poller thread, any blocking on writes will >>> delay the polling - unless you use non-blocking writes. >>> >>>> This Server/Socket combination has the advantage that it easily >>>> integrates >>>> into the signal/slot mechanism of Qt. >>> >>> I think it's good advice, though I wouldn't personally have chosen Qt. >>> >>> This application is not demanding enough to need a shared memory >>> solution, with the attendant need for locks and especially, >>> memory barriers, of which most respondents seem ill-educated. >> >> That's only a problem if you "roll your own" locks. IME, the SysV >> shared memory operations also carry along the SysV semaphores. > > As I said, ill-educated. > > As you'll find out when the SysV semaphore grants you access to > some shared memory resource that has been written, but only some > of the writes have been flushed.I'd appreciate a definitive reference backing your claim.> If you're on hardware that can do this, you need to know about it, > or leave the work to someone else who does... like the authors of > SQLite.And, the folks authoring the semaphore functions *don't*?
Reply by ●June 24, 20162016-06-24
Clifford Heath wrote:> Reinhardt, do you deal with write-blocking if the reader fails? > If the write is in the poller thread, any blocking on writes will > delay the polling - unless you use non-blocking writes.Qt internally uses non-blocking read and write. So this is not a problem.> >> This Server/Socket combination has the advantage that it easily >> integrates into the signal/slot mechanism of Qt. > > I think it's good advice, though I wouldn't personally have chosen Qt. > > This application is not demanding enough to need a shared memory > solution, with the attendant need for locks and especially, > memory barriers, of which most respondents seem ill-educated. > In particular, Mel Wilson's "one writer, many reader" is no > longer so simple with modern multi-core architectures. Writes > do not have to happen in the order they are written, so readers > can get confused.Because of that I prefer to avoid all these locking if it is not needed. Having the data at one place controlled by one thread and send it from there to interested clients via some IPC mechanism make live much easier. And it is also a very effective encapsulation. The other can not even know, how it handled internally in the server. I would use shared memory or similar close coupled methods only if necessary for performance or other requirements. -- Reinhardt
Reply by ●June 24, 20162016-06-24
On 24/06/16 14:27, Don Y wrote:> On 6/23/2016 8:25 PM, Clifford Heath wrote: >> On 24/06/16 10:19, Don Y wrote: >>> On 6/23/2016 4:45 PM, Clifford Heath wrote: >>>> On 23/06/16 23:05, Reinhardt Behm wrote: >>>>> Jack wrote: >>>>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>>>> I'm going to develop a local/remote control of an electronic >>>>>>> device. It >>>>>>> communicates through a RS485 link. >>>>>>> I'm thinking to use SQLite database to store the data. The poller >>>>>>> writes >>>>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>>>> care >>>>>>> the multi-thread/multi-process scenario. >>>> >>>> SQLite "takes care of" that by using a single giant lock. >>>> That might work for you, since it sounds like you probably >>>> have very easy throughput and latency demands. It would >>>> be a good solution if you weren't using Qt. Les Cargill >>>> hasn't seen how remarkably lightweight SQLite actually is. >>>> >>>>> Since you already using Qt, I can propose a solution which I use quite >>>>> often. It is even flying in more than 100 helicopters. Yes, penguins >>>>> can fly >>>>> The poller fetches that data from the remote device via RS485 and >>>>> keeps a >>>>> for it local copy. If permanent storage is a requirement it can also >>>>> take >>>>> care of this. >>>>> In addition it open a QLocalServer which listens for connections from >>>>> other >>>>> processes. >>>>> The other (multiple) processes connect to this server using a >>>>> QLocalSocket. >>>>> They can then poll are are sent the data directly when the poller >>>>> receives >>>>> new data. This way no locking is required since the poller does its >>>>> sending >>>>> in a single thread. >>>> >>>> Reinhardt, do you deal with write-blocking if the reader fails? >>>> If the write is in the poller thread, any blocking on writes will >>>> delay the polling - unless you use non-blocking writes. >>>> >>>>> This Server/Socket combination has the advantage that it easily >>>>> integrates >>>>> into the signal/slot mechanism of Qt. >>>> >>>> I think it's good advice, though I wouldn't personally have chosen Qt. >>>> >>>> This application is not demanding enough to need a shared memory >>>> solution, with the attendant need for locks and especially, >>>> memory barriers, of which most respondents seem ill-educated. >>> >>> That's only a problem if you "roll your own" locks. IME, the SysV >>> shared memory operations also carry along the SysV semaphores. >> >> As I said, ill-educated. >> >> As you'll find out when the SysV semaphore grants you access to >> some shared memory resource that has been written, but only some >> of the writes have been flushed. > > I'd appreciate a definitive reference backing your claim. > >> If you're on hardware that can do this, you need to know about it, >> or leave the work to someone else who does... like the authors of >> SQLite. > > And, the folks authoring the semaphore functions *don't*?The semaphore functions are not guaranteed to flush CPU caches or prevent out-of-order execution of writes to the shared memory. Why would they? The semaphore semantics are not intertwined with the memory semantics. It's common for entry to the kernel to flush the cache, but you need to know - it doesn't happen by magic. But out-of-order execution can still spoil your day. In some architectures, even a single "increment-memory" instruction can allow another CPU to observe the memory cell to be half-incremented. It's all very troubling...
Reply by ●June 24, 20162016-06-24
On 6/23/2016 9:40 PM, Clifford Heath wrote:> On 24/06/16 14:27, Don Y wrote: >> On 6/23/2016 8:25 PM, Clifford Heath wrote: >>> On 24/06/16 10:19, Don Y wrote: >>>> On 6/23/2016 4:45 PM, Clifford Heath wrote: >>>>> On 23/06/16 23:05, Reinhardt Behm wrote: >>>>>> Jack wrote: >>>>>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>>>>> I'm going to develop a local/remote control of an electronic >>>>>>>> device. It >>>>>>>> communicates through a RS485 link. >>>>>>>> I'm thinking to use SQLite database to store the data. The poller >>>>>>>> writes >>>>>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>>>>> care >>>>>>>> the multi-thread/multi-process scenario. >>>>> >>>>> SQLite "takes care of" that by using a single giant lock. >>>>> That might work for you, since it sounds like you probably >>>>> have very easy throughput and latency demands. It would >>>>> be a good solution if you weren't using Qt. Les Cargill >>>>> hasn't seen how remarkably lightweight SQLite actually is. >>>>> >>>>>> Since you already using Qt, I can propose a solution which I use quite >>>>>> often. It is even flying in more than 100 helicopters. Yes, penguins >>>>>> can fly >>>>>> The poller fetches that data from the remote device via RS485 and >>>>>> keeps a >>>>>> for it local copy. If permanent storage is a requirement it can also >>>>>> take >>>>>> care of this. >>>>>> In addition it open a QLocalServer which listens for connections from >>>>>> other >>>>>> processes. >>>>>> The other (multiple) processes connect to this server using a >>>>>> QLocalSocket. >>>>>> They can then poll are are sent the data directly when the poller >>>>>> receives >>>>>> new data. This way no locking is required since the poller does its >>>>>> sending >>>>>> in a single thread. >>>>> >>>>> Reinhardt, do you deal with write-blocking if the reader fails? >>>>> If the write is in the poller thread, any blocking on writes will >>>>> delay the polling - unless you use non-blocking writes. >>>>> >>>>>> This Server/Socket combination has the advantage that it easily >>>>>> integrates >>>>>> into the signal/slot mechanism of Qt. >>>>> >>>>> I think it's good advice, though I wouldn't personally have chosen Qt. >>>>> >>>>> This application is not demanding enough to need a shared memory >>>>> solution, with the attendant need for locks and especially, >>>>> memory barriers, of which most respondents seem ill-educated. >>>> >>>> That's only a problem if you "roll your own" locks. IME, the SysV >>>> shared memory operations also carry along the SysV semaphores. >>> >>> As I said, ill-educated. >>> >>> As you'll find out when the SysV semaphore grants you access to >>> some shared memory resource that has been written, but only some >>> of the writes have been flushed. >> >> I'd appreciate a definitive reference backing your claim. >> >>> If you're on hardware that can do this, you need to know about it, >>> or leave the work to someone else who does... like the authors of >>> SQLite. >> >> And, the folks authoring the semaphore functions *don't*? > > The semaphore functions are not guaranteed to flush CPU caches > or prevent out-of-order execution of writes to the shared memory. > Why would they? The semaphore semantics are not intertwined with > the memory semantics.My mutex functions do exactly that -- to ensure the synchronization primitive works regardless of instruction reordering, the presence of multiple cores, etc. The whole point is to make it possible for a piece of code to be migrated to a different platform (with different hardware capabilities) WITHOUT having to reexamine each potential problem area. I have no idea how the Linux kernel implements the SysV semaphores (or POSIX semaphores). I will have to look at the FreeBSD and NetBSD sources to see how *they* do. I'd be surprised if a kernel capable of running on such hardware would neglect these sorts of things.> It's common for entry to the kernel to flush the cache, but you > need to know - it doesn't happen by magic. But out-of-order > execution can still spoil your day. > > In some architectures, even a single "increment-memory" instruction > can allow another CPU to observe the memory cell to be half-incremented. > It's all very troubling...All the more reason to embed these guarantees in these library functions.
Reply by ●June 24, 20162016-06-24
On 24/06/16 15:20, Don Y wrote:> On 6/23/2016 9:40 PM, Clifford Heath wrote: >> On 24/06/16 14:27, Don Y wrote: >>> On 6/23/2016 8:25 PM, Clifford Heath wrote: >>>> On 24/06/16 10:19, Don Y wrote: >>>>> On 6/23/2016 4:45 PM, Clifford Heath wrote: >>>>>> On 23/06/16 23:05, Reinhardt Behm wrote: >>>>>>> Jack wrote: >>>>>>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>>>>>> I'm going to develop a local/remote control of an electronic >>>>>>>>> device. It >>>>>>>>> communicates through a RS485 link. >>>>>>>>> I'm thinking to use SQLite database to store the data. The poller >>>>>>>>> writes >>>>>>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>>>>>> care >>>>>>>>> the multi-thread/multi-process scenario. >>>>>> >>>>>> SQLite "takes care of" that by using a single giant lock. >>>>>> That might work for you, since it sounds like you probably >>>>>> have very easy throughput and latency demands. It would >>>>>> be a good solution if you weren't using Qt. Les Cargill >>>>>> hasn't seen how remarkably lightweight SQLite actually is. >>>>>> >>>>>>> Since you already using Qt, I can propose a solution which I use >>>>>>> quite >>>>>>> often. It is even flying in more than 100 helicopters. Yes, penguins >>>>>>> can fly >>>>>>> The poller fetches that data from the remote device via RS485 and >>>>>>> keeps a >>>>>>> for it local copy. If permanent storage is a requirement it can also >>>>>>> take >>>>>>> care of this. >>>>>>> In addition it open a QLocalServer which listens for connections >>>>>>> from >>>>>>> other >>>>>>> processes. >>>>>>> The other (multiple) processes connect to this server using a >>>>>>> QLocalSocket. >>>>>>> They can then poll are are sent the data directly when the poller >>>>>>> receives >>>>>>> new data. This way no locking is required since the poller does its >>>>>>> sending >>>>>>> in a single thread. >>>>>> >>>>>> Reinhardt, do you deal with write-blocking if the reader fails? >>>>>> If the write is in the poller thread, any blocking on writes will >>>>>> delay the polling - unless you use non-blocking writes. >>>>>> >>>>>>> This Server/Socket combination has the advantage that it easily >>>>>>> integrates >>>>>>> into the signal/slot mechanism of Qt. >>>>>> >>>>>> I think it's good advice, though I wouldn't personally have chosen >>>>>> Qt. >>>>>> >>>>>> This application is not demanding enough to need a shared memory >>>>>> solution, with the attendant need for locks and especially, >>>>>> memory barriers, of which most respondents seem ill-educated. >>>>> >>>>> That's only a problem if you "roll your own" locks. IME, the SysV >>>>> shared memory operations also carry along the SysV semaphores. >>>> >>>> As I said, ill-educated. >>>> >>>> As you'll find out when the SysV semaphore grants you access to >>>> some shared memory resource that has been written, but only some >>>> of the writes have been flushed. >>> >>> I'd appreciate a definitive reference backing your claim. >>> >>>> If you're on hardware that can do this, you need to know about it, >>>> or leave the work to someone else who does... like the authors of >>>> SQLite. >>> >>> And, the folks authoring the semaphore functions *don't*? >> >> The semaphore functions are not guaranteed to flush CPU caches >> or prevent out-of-order execution of writes to the shared memory. >> Why would they? The semaphore semantics are not intertwined with >> the memory semantics. > > My mutex functions do exactly that -- to ensure the synchronization > primitive works regardless of instruction reordering, the presence > of multiple cores, etc. The whole point is to make it possible for > a piece of code to be migrated to a different platform (with different > hardware capabilities) WITHOUT having to reexamine each potential > problem area.You're still missing the point. The synchronisation primitives can fulfil their contract perfectly - with regard to the semaphore operations - but the memory write behaviour is not part of that contract. Only the semaphore operations are in the contract, and nothing about that says that all memory accesses on both sides must be coherent across all processor cores. Your mutex functions perfectly, I'm sure - but do they guarantee that memory operations either side of the mutex - and not referencing the mutex's memory - is coherent across multiple cores. Note that I am not talking about the memory used to implement the mutex itself, but other memory.> I have no idea how the Linux kernel implements the SysV semaphores > (or POSIX semaphores). I will have to look at the FreeBSD and NetBSD > sources to see how *they* do. I'd be surprised if a kernel capable of > running on such hardware would neglect these sorts of things. > >> It's common for entry to the kernel to flush the cache, but you >> need to know - it doesn't happen by magic. But out-of-order >> execution can still spoil your day. >> >> In some architectures, even a single "increment-memory" instruction >> can allow another CPU to observe the memory cell to be half-incremented. >> It's all very troubling... > > All the more reason to embed these guarantees in these library functions.
Reply by ●June 24, 20162016-06-24
On 6/24/2016 12:19 AM, Clifford Heath wrote:> On 24/06/16 15:20, Don Y wrote: >> On 6/23/2016 9:40 PM, Clifford Heath wrote: >>> On 24/06/16 14:27, Don Y wrote: >>>> On 6/23/2016 8:25 PM, Clifford Heath wrote: >>>>> On 24/06/16 10:19, Don Y wrote: >>>>>> On 6/23/2016 4:45 PM, Clifford Heath wrote: >>>>>>> On 23/06/16 23:05, Reinhardt Behm wrote: >>>>>>>> Jack wrote: >>>>>>>>> Il giorno giovedì 23 giugno 2016 13:52:19 UTC+2, pozz ha scritto: >>>>>>>>>> I'm going to develop a local/remote control of an electronic >>>>>>>>>> device. It >>>>>>>>>> communicates through a RS485 link. >>>>>>>>>> I'm thinking to use SQLite database to store the data. The poller >>>>>>>>>> writes >>>>>>>>>> the database, HTTP and QT reads from it. It seems SQLite will take >>>>>>>>>> care >>>>>>>>>> the multi-thread/multi-process scenario. >>>>>>> >>>>>>> SQLite "takes care of" that by using a single giant lock. >>>>>>> That might work for you, since it sounds like you probably >>>>>>> have very easy throughput and latency demands. It would >>>>>>> be a good solution if you weren't using Qt. Les Cargill >>>>>>> hasn't seen how remarkably lightweight SQLite actually is. >>>>>>> >>>>>>>> Since you already using Qt, I can propose a solution which I use >>>>>>>> quite >>>>>>>> often. It is even flying in more than 100 helicopters. Yes, penguins >>>>>>>> can fly >>>>>>>> The poller fetches that data from the remote device via RS485 and >>>>>>>> keeps a >>>>>>>> for it local copy. If permanent storage is a requirement it can also >>>>>>>> take >>>>>>>> care of this. >>>>>>>> In addition it open a QLocalServer which listens for connections >>>>>>>> from >>>>>>>> other >>>>>>>> processes. >>>>>>>> The other (multiple) processes connect to this server using a >>>>>>>> QLocalSocket. >>>>>>>> They can then poll are are sent the data directly when the poller >>>>>>>> receives >>>>>>>> new data. This way no locking is required since the poller does its >>>>>>>> sending >>>>>>>> in a single thread. >>>>>>> >>>>>>> Reinhardt, do you deal with write-blocking if the reader fails? >>>>>>> If the write is in the poller thread, any blocking on writes will >>>>>>> delay the polling - unless you use non-blocking writes. >>>>>>> >>>>>>>> This Server/Socket combination has the advantage that it easily >>>>>>>> integrates >>>>>>>> into the signal/slot mechanism of Qt. >>>>>>> >>>>>>> I think it's good advice, though I wouldn't personally have chosen >>>>>>> Qt. >>>>>>> >>>>>>> This application is not demanding enough to need a shared memory >>>>>>> solution, with the attendant need for locks and especially, >>>>>>> memory barriers, of which most respondents seem ill-educated. >>>>>> >>>>>> That's only a problem if you "roll your own" locks. IME, the SysV >>>>>> shared memory operations also carry along the SysV semaphores. >>>>> >>>>> As I said, ill-educated. >>>>> >>>>> As you'll find out when the SysV semaphore grants you access to >>>>> some shared memory resource that has been written, but only some >>>>> of the writes have been flushed. >>>> >>>> I'd appreciate a definitive reference backing your claim. >>>> >>>>> If you're on hardware that can do this, you need to know about it, >>>>> or leave the work to someone else who does... like the authors of >>>>> SQLite. >>>> >>>> And, the folks authoring the semaphore functions *don't*? >>> >>> The semaphore functions are not guaranteed to flush CPU caches >>> or prevent out-of-order execution of writes to the shared memory. >>> Why would they? The semaphore semantics are not intertwined with >>> the memory semantics. >> >> My mutex functions do exactly that -- to ensure the synchronization >> primitive works regardless of instruction reordering, the presence >> of multiple cores, etc. The whole point is to make it possible for >> a piece of code to be migrated to a different platform (with different >> hardware capabilities) WITHOUT having to reexamine each potential >> problem area. > > You're still missing the point. The synchronisation primitives can > fulfil their contract perfectly - with regard to the semaphoreHow? If threadX and threadY (operating on the same processor *or* on different cores) can see different orderings, then how is the contract satisfied? It's not an "increment memory" operation...> operations - but the memory write behaviour is not part of that > contract. Only the semaphore operations are in the contract, and > nothing about that says that all memory accesses on both sides > must be coherent across all processor cores.What value is a synchronization primitive if it doesn't provide synchronization? How is it fulfilling its contract if it *doesn't*? All synchronization operations MUST perform barriers to memory and operator reordering.> Your mutex functions perfectly, I'm sure - but do they guarantee > that memory operations either side of the mutex - and not referencing > the mutex's memory - is coherent across multiple cores. Note that > I am not talking about the memory used to implement the mutex > itself, but other memory.Yes -- as they include memory barriers specific to the particular processor architecture involved. The whole point is NOT to force the developer to have to wonder whether it's a single-threaded environment or multithreaded; uniprocessor or multiprocessor. The function call is a fence. From a quick peek through some of the online Linux sources, their SysV semaphores also seem to operate in a similar manner. I've not yet looked through the *BSD sources... Are you claiming the compiler can reorder instructions *across* an opaque system call? E.g., decide to invoke the code at main() AFTER the code at exit()?? (what's to prevent this??)>> I have no idea how the Linux kernel implements the SysV semaphores >> (or POSIX semaphores). I will have to look at the FreeBSD and NetBSD >> sources to see how *they* do. I'd be surprised if a kernel capable of >> running on such hardware would neglect these sorts of things. >> >>> It's common for entry to the kernel to flush the cache, but you >>> need to know - it doesn't happen by magic. But out-of-order >>> execution can still spoil your day. >>> >>> In some architectures, even a single "increment-memory" instruction >>> can allow another CPU to observe the memory cell to be half-incremented. >>> It's all very troubling... >> >> All the more reason to embed these guarantees in these library functions. >
Reply by ●June 24, 20162016-06-24
On 6/23/2016 9:39 PM, Reinhardt Behm wrote:>> This application is not demanding enough to need a shared memory >> solution, with the attendant need for locks and especially, >> memory barriers, of which most respondents seem ill-educated. >> In particular, Mel Wilson's "one writer, many reader" is no >> longer so simple with modern multi-core architectures. Writes >> do not have to happen in the order they are written, so readers >> can get confused. > > Because of that I prefer to avoid all these locking if it is not needed.You're just hiding by wrapping it in an IPC mechanism.> Having the data at one place controlled by one thread and send it from there > to interested clients via some IPC mechanism make live much easier. And it > is also a very effective encapsulation. The other can not even know, how it > handled internally in the server.Yes, but it is also considerably more expensive. Every consumer has to be informed of every update to the data in which it has "expressed an interest" (assuming you don't broadcast ALL data to ALL consumers). E.g., I install triggers in my RDBMS so that any update to a particular (set of) parameter(s) results in an up-call notification of the interested parties that might want to *see* that new value (so, I don't have to broadcast all updates of all data to all consumers). But, that upcall carries considerable cost (from trigger through upcall to client notification and eventual re-query of the RDBMS). It can effectively bottleneck the RDBMS's operation (cuz another up-call could be enqueued right behind the first one even before the first has finished!). So, the nature of the problem changes -- to one of deciding *what* information is "genuinely of interest" to each client... AND, how timely that notification should be made (if you can defer a notification, then you might be able to eliminate it in favor of a subsequent notification) E.g., if some client (having permission to do so!) updates the current timezone, the wall-clock display should *probably* be updated to reflect that. OTOH, it probably can wait a second or two. And, if the timezone is updated AGAIN before the notification is dispatched, the first update can safely be ignored in favor of the second. OTOH, if *every* update must be conveyed to the client(s), then the notification effectively becomes part of the update transaction and limits how frequently the value can be updated.> I would use shared memory or similar close coupled methods only if necessary > for performance or other requirements.IME, people are more accustomed to using shared memory solutions as this mimics the familiar interplay of foreground/background processing; people *seem* to know that some coordination must exist between an ISR and its "clients". OTOH, sharing memory in an environment with individual, protected memory spaces can be intimidating the first time.
Reply by ●June 24, 20162016-06-24
Il 23/06/2016 15:47, Mel Wilson ha scritto:> On Thu, 23 Jun 2016 13:52:21 +0200, pozz wrote: > >> I'm new to embedded Linux so this question could be very simple for many >> of you. Most probably, it isn't directly related to embedded world, but >> to Linux OS generally. Anyway I think it is a common scenario in >> embedded applications. >> >> I'm going to develop a local/remote control of an electronic device. It >> communicates through a RS485 link. >> The local control will be a touch-screen display (I'm going to use QT >> graphic libraries). >> The remote control will be HTTP (web server). >> >> I think a good approach will be to develop a simple application, the >> poller, that communicates with the electronic device and implements the >> RS485 protocol. The poller continuously acquires the current >> status/settings of the device and store them in some "shared" way. >> >> The graphic application (QT-based) and the web server (CGI) should >> access to the data retrieved by the poller. >> >> What is the best method to share the data generated by an application >> (the poller) among two or more applications (QT and CGI)? >> In this scenario, I think it's important to lock the "shared data" >> before accessing them (reading or writing), in order to avoid reading >> incoerent data. Indeed, if the poller writes the data at the same time >> (Linux OS is multi-tasking) the web server reads them, they could be >> incoerent. >> >> I'm thinking to use SQLite database to store the data. The poller writes >> the database, HTTP and QT reads from it. It seems SQLite will take care >> the multi-thread/multi-process scenario. >> >> Any suggestions? > > I did that with System V IPC, using message passing for controlYes, another problem to solve is the other way: QT and HTTP server could send some control commands to the electronic device (switch on, switch off, change this...).> and > shared memory blocks for info common to all the processes. I was able to > avoid any locking using the one-writer/many-readers trick.It is exactly what I will have: one writer (the poller that will continuously fetch updated data from the device) and some readers (at least, HTTP server and QT graphic libraries). Doesn't it needed a syncronization mechanism (semaphore) in this scenario, using shared memory to share common data?> The code was > more low-level than many people, perhaps, would like, but it worked well. > > Mel. >
