Use of volatile in structure definitions to force word accesses

John Devereux <john@devereux.me.uk> wrote:

> The *C language* standard may allow it, but on a particular machine
> architecture there can be an ABI that specifies these things.

Yep, but why learn bad habits that will break your code when you change 
platforms in the next project?

-a

On 2015-09-24, Anders.Montonen@kapsi.spam.stop.fi.invalid <Anders.Montonen@kapsi.spam.stop.fi.invalid> wrote:
> John Devereux <john@devereux.me.uk> wrote:
>
>> The *C language* standard may allow it, but on a particular machine
>> architecture there can be an ABI that specifies these things.
>
> Yep, but why learn bad habits that will break your code when you change 
> platforms in the next project?

Even if a particular C compiler's manual says that adding "volatile"
to a struct member will do something-or-other to the access width (and
testing with today's version compiler on today's host with today's
options validates that claim), I've found over the decades that
relying on features like that ends up causing far more trouble than it
avoids.

Sure, it seems cool now. But, a few years from now when it stops
working and it takes somebody two months to find the cause of the
mysterious data corruptions -- you don't want to bump into that
somebody in a dark alley.

-- 
Grant Edwards               grant.b.edwards        Yow! All this time I've
                                  at               been VIEWING a RUSSIAN
                              gmail.com            MIDGET SODOMIZE a HOUSECAT!

On 2015-09-24, David Brown <david.brown@hesbynett.no> wrote:
> On 24/09/15 01:12, Simon Clubley wrote:
>> 
>> The cross compiler I am using is gcc 4.5.{something} although, based
>> on the other problem reports I have come across, I suspect it's still
>> a problem in at least some newer versions as well.
>> 
>
> That explains it - the issue was fixed in 4.6.
>

Interesting.

One of the things which has clearly thrown me here is that I went and
looked at the comp.lang.ada thread from a few weeks ago before commenting
here about bitfields.

In that thread, the OP is using a reasonably recent version of GNAT
(the name for the Ada gcc front end) from ACT. The last time I checked,
ACT were using gcc 4.7 for their own internal branch of the gcc tree
so I thought the problem was present upto at least gcc 4.7.

However, it's possible that the Ada front end is still exposing the
problem in a way that the C front end no longer is.

Simon.

-- 
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

On 2015-09-24, David Brown <david.brown@hesbynett.no> wrote:
>
> If you want both assignments to be done in a single RMW operation, you
> must do so explicitly - that's what the "all" field is for.
>
> void test2(void) {
>     periphRegister_t cache;
>     cache.all = periphRegister.all;
>     cache.bit0 = 1;
>     cache.bit11 = 0;
>     periphRegister.all = cache.all;
> }
>

You have to do this in Ada as well at the moment.

There's currently an Ada Issue which talks about adding the ability
to the next version of Ada to list the bitfields to be updated in
a single assignment statement so that the Ada version of the above
would become a single RMW sequence without having to use a temporary
variable.

It's here if you are interested:

http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0127-1.txt?rev=1.1

Simon.

-- 
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

Am 23.09.2015 um 21:55 schrieb Tim Wescott:

> I would REALLY like one of the compiler guys to weigh in on this, because
> I'm wondering if collecting all those unions into a struct and calling it
> volatile isn't forcing a 32-bit access.

As far as the language definition is concerned, "volatile" is not really 
supposed to have anything to do with that.

Volatile specifies that all accesses to an object have to actually 
happen, in the order specified by the source.  So no optimize-out of 
reads or writes, and no shuffling of volatile accesses among each other, 
among other restrictions.

But it says nothing at all about the width of accesses.  In th 
Standardese language, the reason is that it's "implementation-defined 
what constitutes access to a volatile-qualified object".  One of the 
meanings of that rather opaque statement is that they left open whether 
access to a single element of the compound also constitutes an "access" 
to the whole thing.  And only such accesses would be protected by the 
specified effect of "volatile".  So no, the outer "volatile" is not 
really specified to have any effect on referral to the inner elements.

The only way I can see to somewhat portably control access width is to 
do away with the all the structs and bitfields altogether, and actually 
use bitwise operators on volatile-qualified object of the necessary 
size.  I.e. if you write

	volatile uint32_t *pointer_to_register = some_address;
	*pointer_to_register |= some_mask;

then that _does_ force the compiler to actually do a 32-bit access, 
because the object you're accessing is, beyond reasonable doubt, 
qualified "volatile", so the rules do apply here: this object has to be 
read, and it has to be written.

Bit fields can not portably achieve the same effect, because access to 
them does not necessarily constitute access to the containing compound 
data structure.

On 24/09/15 21:08, Tim Wescott wrote:
> On Thu, 24 Sep 2015 15:28:59 +0200, David Brown wrote:
>
>> On 24/09/15 01:12, Simon Clubley wrote:
>>>
>>> The cross compiler I am using is gcc 4.5.{something} although, based on
>>> the other problem reports I have come across, I suspect it's still a
>>> problem in at least some newer versions as well.
>>>
>>>
>> That explains it - the issue was fixed in 4.6.
>>
>> There are a few questionable issues in later versions of the compiler,
>> in regard to particularly complex bitfield structures.  The trouble here
>> is that volatile accesses are not well defined in C - much of it is left
>> as "implementation defined" (meaning the compiler can choose, but it
>> should be consistent and preferably documented) or even "unspecified"
>> (meaning it has to follow certain behaviour patterns, but the compiler
>> can choose which to use - and it can choose arbitrarily from case to
>> case).
>>
>> As with all uses of volatile, if you try to make the code unreasonably
>> complicated, you run the risk of code that does not do as you hoped -
>> usually because you don't understand it yourself, but sometimes because
>> of imprecise standards definitions and sometimes because of compiler
>> bugs.
>
> Or, as I alluded to in my original post, you run the risk of making code
> that's complicated to confuse the optimizer enough that it throws up its
> hands and does what you want at the moment, without guaranteeing that
> future optimizers wouldn't be able to unwind what you did and break it by
> making it "right".
>
> Which is why I was asking about the intent of the volatile keyword WRT
> unions.
>

The intent is, I think, pretty clear - a volatile access is the same 
regardless of whether or not a union is involved.  The issue with 
volatile bitfield access was that the size of the access was not well 
defined (the standards don't give any clues), and for many purposes the 
use of differently sized accesses can give significantly more efficient 
code that is perfectly correct on that platform.

There are, I think, three main issues with regard to getting volatile to 
work as the programmer intended.  If you understand these, you should be 
fine.

1. Programmers often don't understand volatile.  Some believe it makes 
accesses atomic - so that on an 8-bit micro, they confuse making a 
uint16_t volatile with making it safe for swapping data between contexts 
(different threads or interrupt functions).  And some believe that 
volatile accesses enforce certain orders on non-volatile accesses, or 
calculations.

2. The standards are unclear - "what constitutes a volatile access is 
implementation dependent".  If v is a volatile variable, and your target 
supports arithmetic operation directly on memory addresses, then v++ may 
do a read, modify, then write, or it may do a single add to the memory 
address.  If you ask to read a volatile variable of a certain size, then 
the compiler may read it in multiple parts, or it may read additional 
data beside the actual variable, or it may read only part of the 
variable (if it only needs part of it).  Unless your compiler makes 
guarantees in this area (such as gcc's -fstrict-volatile-bitfields), you 
don't know.

3. Compilers may have bugs.  The typical case is when complex volatile 
operations interact with complex optimisations, and the compiler does 
code movement or simplification that is not allowed for volatiles.

A key way to minimise the risks of anything going on is to keep 
statements and expressions involving volatiles relatively simple.  Try 
to stick to a single read or a single write of a volatile at a time. 
Then the code is clear to a human reader, and clear to the compiler. 
(Having the volatile item deep within nested structs and unions is not a 
problem, nor does it matter if the volatile qualifier is on the 
innermost item, the outermost structure, a cast wrapper on the outside, 
or anything inbetween.)  If vp is a volatile pointer to a volatile int, 
and vi is a volatile int, with foo() being a function, then writing 
"vp[vi = foo()]++" may be legal C, but it is asking for trouble.

On 24/09/15 22:32, Simon Clubley wrote:
> On 2015-09-24, David Brown <david.brown@hesbynett.no> wrote:
>> On 24/09/15 01:12, Simon Clubley wrote:
>>>
>>> The cross compiler I am using is gcc 4.5.{something} although, based
>>> on the other problem reports I have come across, I suspect it's still
>>> a problem in at least some newer versions as well.
>>>
>>
>> That explains it - the issue was fixed in 4.6.
>>
>
> Interesting.
>
> One of the things which has clearly thrown me here is that I went and
> looked at the comp.lang.ada thread from a few weeks ago before commenting
> here about bitfields.
>
> In that thread, the OP is using a reasonably recent version of GNAT
> (the name for the Ada gcc front end) from ACT. The last time I checked,
> ACT were using gcc 4.7 for their own internal branch of the gcc tree
> so I thought the problem was present upto at least gcc 4.7.
>
> However, it's possible that the Ada front end is still exposing the
> problem in a way that the C front end no longer is.
>
> Simon.
>

I am afraid I can't answer for GNAT - the tiny amount of Ada that I have 
tried does not touch on this sort of thing.

One other point is that only some targets have 
-fstrict-volatile-bitfield enabled by default (ARM, for example) - on 
some targets, you need to enable it manually if you want that behaviour.

On 2015-09-24, David Brown <david.brown@hesbynett.no> wrote:
>
> I am afraid I can't answer for GNAT - the tiny amount of Ada that I have 
> tried does not touch on this sort of thing.
>
> One other point is that only some targets have 
> -fstrict-volatile-bitfield enabled by default (ARM, for example) - on 
> some targets, you need to enable it manually if you want that behaviour.
>

Yes, the OP in that thread was using ARM. I'm assuming there's something
in the Ada front end which is causing this to still be an issue for
Ada code.

Simon.

-- 
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

On 24/09/15 23:47, Hans-Bernhard Bröker wrote:
> Am 23.09.2015 um 21:55 schrieb Tim Wescott:
>
>> I would REALLY like one of the compiler guys to weigh in on this, because
>> I'm wondering if collecting all those unions into a struct and calling it
>> volatile isn't forcing a 32-bit access.
>
> As far as the language definition is concerned, "volatile" is not really
> supposed to have anything to do with that.
>
> Volatile specifies that all accesses to an object have to actually
> happen, in the order specified by the source.  So no optimize-out of
> reads or writes, and no shuffling of volatile accesses among each other,
> among other restrictions.
>
> But it says nothing at all about the width of accesses.  In th
> Standardese language, the reason is that it's "implementation-defined
> what constitutes access to a volatile-qualified object".  One of the
> meanings of that rather opaque statement is that they left open whether
> access to a single element of the compound also constitutes an "access"
> to the whole thing.  And only such accesses would be protected by the
> specified effect of "volatile".  So no, the outer "volatile" is not
> really specified to have any effect on referral to the inner elements.
>

I am not sure of that final point in regard to the specifications - but 
I am very sure of how it works in all compilers I have seen.  A volatile 
qualifier on a struct, union or array applies equally to all members of 
that struct, union or array (the same applies to the const qualifier). 
So there is no difference between:

struct { volatile uint32_t a; volatile uint32_t b; } s1;

and

volatile struct { uint32_t a; uint32_t b; } s2;


> The only way I can see to somewhat portably control access width is to
> do away with the all the structs and bitfields altogether, and actually
> use bitwise operators on volatile-qualified object of the necessary
> size.  I.e. if you write
>
>      volatile uint32_t *pointer_to_register = some_address;
>      *pointer_to_register |= some_mask;
>
> then that _does_ force the compiler to actually do a 32-bit access,
> because the object you're accessing is, beyond reasonable doubt,
> qualified "volatile", so the rules do apply here: this object has to be
> read, and it has to be written.

No, that does not force the compiler to use 32-bit accesses.  It forces 
the compiler to access all parts of the 32-bit object, but the compiler 
is free to divide it into 4 8-bit accesses if it wants, and it can order 
those accesses as it likes.  It is perfectly acceptable (to the 
standards) for the compiler to do 4 byte-sized RMW operations - or 4 
byte-sized reads, followed by the or, then 2 16-bit writes in the 
opposite order.

Of course, a compiler is not going to do that (unless alignment issues 
or the bit-size of the processor forces it to break up the accesses). 
But it could - according to the standards.

You are right that such full-object manipulation is less likely to bump 
into compiler bugs than volatile bitfield operation, but volatile 
bitfield operation is considered reliable enough for most targets and 
most modern mainstream compilers (such as gcc post 4.6) - that is why it 
is a common arrangement for manufacturer-supplied header files.

>
> Bit fields can not portably achieve the same effect, because access to
> them does not necessarily constitute access to the containing compound
> data structure.

We are talking here about accessing hardware registers - it is 
inherently non-portable.  Even portability across compilers (for those 
few that now use anything other than gcc for ARM) is unlikely.

Tim Wescott <seemywebsite@myfooter.really> writes:
> I, for one, would really like to see one of our resident compiler experts 
> answer the question at the bottom.

Well, I'm the resident compiler guy who fixed it in gcc :-)

As others have noted, you need the version of gcc that supports
-fstrict-volatile-bitfields, *and* you need to have a well-defined
(i.e. unambigious) struct (no holes, consistent types, etc), *and* you
need to have a port that supports it (ARM does), *and* it has to be a
size/alignment that the hardware supports (duh), *and* the
-fstrict-volatile-bitfields options needs to be enabled (a later version
of gcc has a third setting that means "unless the ABI says otherwise"
but the result is the same *if* your struct is well-defined) (-fs-v-b
might (should?) be the default for ports that support it, I don't recall
if they argued to change that).

So a struct like this should be OK:

volatile struct foo {
  int32_t a:17;
  int32_t b:5;
  int32_t c:10;
};

but this would not be:

volatile struct foo {
  char a:8;
  int32_t b:24;
};

But, the only portable reliable way to guarantee the right type of
access is to not use bitfields.