EmbeddedRelated.com
Forums
The 2026 Embedded Online Conference

Languages, is popularity dominating engineering?

Started by Ed Prochak December 12, 2014
Am 13.12.2014 um 17:36 schrieb upsidedown@downunder.com:

> The problem with many small 8 bitters is that they do not have good > stack pointer relative addressing modes.
Or, like the 8051, none at all, which is why compilers for that platform default to not using the stack for automatic variables. There's not enough of it, and it's just too darn painful to use. But that's for the compiler writer to figure out how to deal with. Even in worst case the resulting code will never be less memory efficient than a strategy of "just make everything static".
Am 13.12.2014 um 21:22 schrieb Les Cargill:
> Hans-Bernhard Br�ker wrote:
>> Making variables static when they don't need to be blows up memory consumption >> considerably.
> This varies.
Not really. Supposing you start out with a correct program and compiler, making previously automatic variables static can never reduce memory consumption. It can only increase it.
> This being said, I haven't seen an 8 bit micro in some time. Even for > PIC, they've been 16 or 32 bit and not all that memory constrained.
Careful you don't fall for Microchip's marketing ploys too easily, there. Those 32-bit "PICs" have less of a meaningful relation to the original PIC than the doomed Itanium had with the 8086.
On 12/13/2014 5:21 PM, Hans-Bernhard Br�ker wrote:

> Careful you don't fall for Microchip's marketing ploys too easily, > there. Those 32-bit "PICs" have less of a meaningful relation to the > original PIC than the doomed Itanium had with the 8086. >
LOL, is this news ??? The PIC32 has a MIPS M4K Core. http://www.microchip.com/pagehandler/en-us/family/32bit/architecture-pic32mxfamily.html#utm_medium=Press-Release&utm_term=PIC32MX1-2-5-LasVegas_PR_11-3-14&utm_content=MCU32&utm_campaign=PIC32MX+Page
Don Y wrote:
> On 12/13/2014 1:48 PM, Les Cargill wrote: >> Don Y wrote: >>> Hi Simon, >>> >>> On 12/13/2014 8:49 AM, Simon Clubley wrote: >>>> On 2014-12-13, Hans-Bernhard Br&ouml;ker <HBBroeker@t-online.de> wrote: >>>>> Am 13.12.2014 um 13:56 schrieb Simon Clubley: >>>>>> In my case, pulling them out into .bss means it's easy to look at the >>>>>> linker map and see, at compile time, exactly how much memory is >>>>>> required >>>>>> for the variables making it far easier and reliable to analyze memory >>>>>> usage. >>>>> >>>>> The problem is that it doesn't just make memory consumption easier to >>>>> see ... it also makes it larger than it needs to be. So there's a >>>>> good >>>>> chance you'll run out of memory _because_ you wanted to figure out >>>>> if/when you run out of memory. >>>> >>>> OTOH, it's a lot better than having to deal with subtle memory trashing >>>> errors because your now larger stack has descended into the space >>>> allocated to .bss (or even .data) and you find out the hard way that >>>> your code is now too big for the resources on the MCU you are currently >>>> using. >>> >>> Statics are a real downer if you're writing reentrant code. You have to >>> ("manually") ensure (by design) that no two consumers can access >>> <whatever> >>> is reliant on that static "at the same time". Even if the static isn't >>> "required" to preserve data between function invocations (e.g., like >>> strtok). >>> >>> A developer then needs intimate familiarity with every "library" that he >>> calls upon (i.e., code written by the guy in the next cubicle) to ensure >>> he isn't exposing his code to one of those (typical) "intermittents" >>> that >>> you never manage to track down (because its impractical to reproduce the >>> EXACT conditions that caused it to manifest). >> >> I just wonder what you guys are doing where you have these problems. :) > > Simple: dealing with others that aren't as > skilled/disciplined/motivated/etc. > Usually, folks from desktop environments where they don't have to worry > much about the code they are writing. > > Over the years, I have learned to write my code so others can't break *it*. > Tired of having to prove that *my* code is working properly -- by finding > the bug in someone *else's* code (e.g., accessing a private struct in > my code that isn't exported by the interface; failing to observe the > contract > for a particular piece of code, etc.) > >> I wouldn't use too many statics in *library* code. This being said, >> the 'C' >> library uses them all over the place. > > Most functions with internal state (e.g., strtok, asctime/localtim, et al.) > have obvious workarounds (foo_r, etc.). > > Still others *obviously* need to be munged to support > reentrancy/multithreaded > use (errno, malloc, signal et al., etc.) > > I've encountered things like printf() that choke badly (due to static > buffers used for conversion). And, even floating point support ("helper > routines") that precluded use in multithreaded environments (i.e., you > cah to treat the state of those helpers AS IF they were registers in an > FPU) > >> Obviously, if you have contention over a memory object/reentrancy >> issues, you >> can't do this. But mainly it's about each ... thread/routine >> suite having its own memory - and not using dynamic memory when you >> don't need >> it. And when in doubt, use a semaphore. > > If the language doesn't inherently (explicitly) support concurrency, those > hooks can be costly. E.g., up to and including a trap. > > If the developer doesn't *know* he's being screwed (because the > interface for > the library doesn't *disclose* this sort of detail!), he won't know to > protect > "shared objects" (and the objects won't know how to protect themselves) > >> Arrange the larger structure of the peice to where things interact >> minimally >> and you'll have no problems. *This* buffer is only used >> for *this* purpose. That's part of the point. > > Then why have it persistent? It only needs to be around for *that* purpose > so let it go away afterwards. >
Because - if you have the memory - it's simpler to just leave it in place. But mainly because a statically allocated buffer is measurable with sizeof(). If you can use malloc()/free() pairs such that you're guaranteed to never leak, then that works too.
>> If you're memory constrained, you just have to use stack or globals. > > Globals are The Root of All (most) Evil.
Nah. Too many is too many. But for say an 8 or 16 bit PIC, you'll be using globals. "globals" includes control block structs, which provides some degree of ... control.
> You always want to control > the exposure of every datum/object.
Ideally, yes.
> E.g., if you (a function/subr) have > to rely on me to give you a pointer (reference) to an object, then you > won't stomp on it without my knowledge of that. > >> But hopefully, the problem to be solved is enough smaller that this >> doesn't render the thing incomprehensible. >> >> Sytstems I've seen lately, each "thread" is pretty much completely >> isolated >> form other "threads" except for a few variables that manage >> interaction. You >> can "prove out" the interface with grep and a piece of paper. When >> that gets >> too messy, you add interface routines. > > I am a huge fan of true isolation (protection domains). It makes coding > and debugging *so* much easier -- step out of line and the OS brings down > the hammer on you! >
Yep.
> But, this is a more costly feature (e.g., processes vs. threads). > > The "easy way out" is to adopt things like C-S relationships between > producers and consumers. *Formalize* their interactions (at some cost > for the interface). >
Absolutely. <snip>
>> That's somewhat unreliable. > > It's not intended to be the cat's meow. It's meant to give you an idea > of what your stack penetration is so you can verify it is on a par with > what you expected *or* completely out of whack. > > When I write ASM code, in addition to describing the call/return interface > in terms of "changes to the machine's state" (registers going in vs. out, > memory altered, etc.) I also quantify the maximum stack penetration (as > a function of inputs, if thusly related) >
Understood - I've done it too.
>>> E.g., during development, my "create_task()" fills the stack with a >>> regular >>> pattern. At each reschedule(), I look at the value of the stack pointer >>> that my context switch will now preserve and: >>> - determine if it is deeper into the stack than any previously recorded >>> instance >>> - if so, store that value (deepest_stack_pointer) and >>> - verify that it is within the range of valid addresses for the memory >>> allocated for the stack (if not, fall into the debugger before the >>> "contamination" spreads, obfuscating the underlying *cause* >>> Then, periodically, explore the region *beyond* the stack pointer >>> to see how much of this "regular pattern" has been obliterated *between* >>> reschedule()'s. >> >> Oy. I think you're explaining exactly why I prefer the way I do it :) >> I'll also make sure that plenty of stack is allocated; over do it. > > I live with tightly constrained resources. It's important for me to size > large objects (e.g., stacks, heaps, etc.) to fit their actual *needs* and > not just "throw memory at them". E.g., my memory allocator lets me "trim" > allocations (instead of being forced to release them in the same chunks > that they were allocated) as well as *extend* existing allocations (the > policy that the allocator uses can be specified in the allocation > request -- e.g., find me a piece of memory that adjoins *this* piece). > > This allows me to "move" memory between a consumer and producer. E.g., > arrange for the producer and consumer's memory to be contiguous and "free" > it from one while "alloc"ing it to the other. >
Memory is cheap.
>>>> After all, code doesn't always remain static and quite often has >>>> functionality added to it, so you may hit the resource limit even with >>>> your stack based approach anyway. >>> >>> You also have to examine your algorithms to verify that their behavior >>> is appropriately bounded. E.g., I rely on recursive algorithms a lot >>> (simple, elegant). But, have to ensure the constraints governing the >>> recursion are known a priori. >> >> Ah, Well, I only use recursion very sparingly if at all. > > The iterative/recursive duality implies you can always avoid it. But, > often iterative solutions are much more difficult to "get right" than > their equivalent recursive solutions (too much manual housekeeping). > >>>> I suppose the major thing driving me here is to use development >>>> techniques >>>> which allow me a better chance to find out about these potential >>>> issues in >>>> a controlled deterministic manner as early on in the development >>>> process >>>> as possible. >>> >>> The first time you have to chase down this sort of "problem" will pay >>> for >>> every precaution you ever take against it in future efforts! :-/ >> >> I haven't seen a stack overflow in decades, excepting where I'm porting >> code. > > Then you haven't been tasked with trimming your stack to fit the needs of > the routines using it! Or your heaps. :> >
Newp - haven't run into that in ages. It's just time-intensive. I'm sitting on about 100K lines of code these days. Maybe more.
> One of the "problems" with "embedded" is it groups a wide variety of > application domains and implementation constraints into a single subject. > E.g., in the same application/codebase, I have devices that use Q10.14 > while others use "Big Rationals" based on the resources available to each. >
I'm just saying - there are $50 ARM systems - COTS - that sport gigs of memory these days.
> An unfortunate consequence of many languages is that you're pretty much > "stuck" with the types that the language gives you. This forces you > to express "other" things in unnatural ways -- that are more prone to > error (e.g., it would be nice to be able to use infix notation on > ints, BCD's, floats, Qs, bigrats, decimals, etc. -- even INTERCHANGEABLY!). > If you're working in a resource rich environment, this isn't an issue. > But, as you get more resource constrained, you tend to *need* to do these > more often *and* have less capabilities to do them > portably/safely/intuitively.
-- Les Cargill
Hans-Bernhard Br&ouml;ker wrote:
> Am 13.12.2014 um 21:48 schrieb Les Cargill: > >> I wouldn't use too many statics in *library* code. This being said, the >> 'C' library uses them all over the place. > > It's really not that bad, actually. Yes, there are a few such things, > but mostly in functions that I, for one, have never felt tempted to use > in embedded code. And most of those have been deprecated by thread-safe > alternatives since back then. > >> Arrange the larger structure of the peice to where things interact >> minimally and you'll have no problems. *This* buffer is only used >> for *this* purpose. > > But it'll still occupy resources even while the program is doing *that*, > which has nothing to with *this* whatsoever. That's where this approach > becomes wasteful. >
Obviously, if you don't have the RAM you have to do something else. But if you have it, a few buffers here and there in... I dunno, dozens of K aren't gonna hurt you on a target with multiple megs of SDRAM. It only has a cost if it *has* a cost. But being able to do away with memory allocation fail is worth something.
>> If you're memory constrained, you just have to use stack or globals. > > I do wonder how you expect the distinction between statics and globals > to have any effect at all regarding memory size constraints. >
True enough. static just controls access. -- Les Cargill
Hi Les,

On 12/13/2014 7:05 PM, Les Cargill wrote:

>>>>>>> In my case, pulling them out into .bss means it's easy to look at the >>>>>>> linker map and see, at compile time, exactly how much memory is >>>>>>> required >>>>>>> for the variables making it far easier and reliable to analyze memory >>>>>>> usage. >>>>>> >>>>>> The problem is that it doesn't just make memory consumption easier to >>>>>> see ... it also makes it larger than it needs to be. So there's a >>>>>> good >>>>>> chance you'll run out of memory _because_ you wanted to figure out >>>>>> if/when you run out of memory. >>>>> >>>>> OTOH, it's a lot better than having to deal with subtle memory trashing >>>>> errors because your now larger stack has descended into the space >>>>> allocated to .bss (or even .data) and you find out the hard way that >>>>> your code is now too big for the resources on the MCU you are currently >>>>> using. >>>> >>>> Statics are a real downer if you're writing reentrant code. You have to >>>> ("manually") ensure (by design) that no two consumers can access >>>> <whatever> >>>> is reliant on that static "at the same time". Even if the static isn't >>>> "required" to preserve data between function invocations (e.g., like >>>> strtok). >>>> >>>> A developer then needs intimate familiarity with every "library" that he >>>> calls upon (i.e., code written by the guy in the next cubicle) to ensure >>>> he isn't exposing his code to one of those (typical) "intermittents" >>>> that >>>> you never manage to track down (because its impractical to reproduce the >>>> EXACT conditions that caused it to manifest). >>> >>> I just wonder what you guys are doing where you have these problems. :) >> >> Simple: dealing with others that aren't as >> skilled/disciplined/motivated/etc. >> Usually, folks from desktop environments where they don't have to worry >> much about the code they are writing. >> >> Over the years, I have learned to write my code so others can't break *it*. >> Tired of having to prove that *my* code is working properly -- by finding >> the bug in someone *else's* code (e.g., accessing a private struct in >> my code that isn't exported by the interface; failing to observe the >> contract >> for a particular piece of code, etc.) >> >>> I wouldn't use too many statics in *library* code. This being said, >>> the 'C' >>> library uses them all over the place. >> >> Most functions with internal state (e.g., strtok, asctime/localtim, et al.) >> have obvious workarounds (foo_r, etc.). >> >> Still others *obviously* need to be munged to support >> reentrancy/multithreaded >> use (errno, malloc, signal et al., etc.) >> >> I've encountered things like printf() that choke badly (due to static >> buffers used for conversion). And, even floating point support ("helper >> routines") that precluded use in multithreaded environments (i.e., you >> cah to treat the state of those helpers AS IF they were registers in an >> FPU) >> >>> Obviously, if you have contention over a memory object/reentrancy >>> issues, you >>> can't do this. But mainly it's about each ... thread/routine >>> suite having its own memory - and not using dynamic memory when you >>> don't need >>> it. And when in doubt, use a semaphore. >> >> If the language doesn't inherently (explicitly) support concurrency, those >> hooks can be costly. E.g., up to and including a trap. >> >> If the developer doesn't *know* he's being screwed (because the >> interface for >> the library doesn't *disclose* this sort of detail!), he won't know to >> protect >> "shared objects" (and the objects won't know how to protect themselves) >> >>> Arrange the larger structure of the peice to where things interact >>> minimally >>> and you'll have no problems. *This* buffer is only used >>> for *this* purpose. That's part of the point. >> >> Then why have it persistent? It only needs to be around for *that* purpose >> so let it go away afterwards. > > Because - if you have the memory - it's simpler to just leave it in place. But > mainly because a statically allocated buffer is measurable with sizeof().
First, you don't always have the memory. As to sizeof: static some_type_t file_scope[SOME_NUMBER]; foo() { static another_type_t function_scope[ANOTHER_NUMBER]; ... } You *know* the sizes of each of these: SOME_NUMBER * sizeof(some_type_t) and ANOTHER_NUMBER * sizeof(another_type_t). Of course, you can also do: sizeof(file_scope) and sizeof(function_scope). Or, derive the number of elements: sizeof(array_name)/sizeof(array_name[0])
> If you can use malloc()/free() pairs such that you're guaranteed to > never leak, then that works too.
I think you are using "static" in a different sense than intended in the standard. :> Regardless, you also know how big each SUCCESSFUL allocation (minimum size thereof) will be just by the arguments to malloc().
>>> If you're memory constrained, you just have to use stack or globals. >> >> Globals are The Root of All (most) Evil. > > Nah. Too many is too many. But for say an 8 or 16 bit PIC, you'll be using > globals.
I don't see how that follows. main() { some_type_t not_a_global[SOMETHING]; ... } works just fine. Leave it to the compiler to put things where it can get at them. *You* decide what sorts of visibility objects should have. By using the narrowest scope possible.
> "globals" includes control block structs, which provides some > degree of ... control. > >> You always want to control >> the exposure of every datum/object. > > Ideally, yes. > >> E.g., if you (a function/subr) have >> to rely on me to give you a pointer (reference) to an object, then you >> won't stomp on it without my knowledge of that. >> >>> But hopefully, the problem to be solved is enough smaller that this >>> doesn't render the thing incomprehensible. >>> >>> Sytstems I've seen lately, each "thread" is pretty much completely >>> isolated >>> form other "threads" except for a few variables that manage >>> interaction. You >>> can "prove out" the interface with grep and a piece of paper. When >>> that gets >>> too messy, you add interface routines.
>>>> E.g., during development, my "create_task()" fills the stack with a >>>> regular >>>> pattern. At each reschedule(), I look at the value of the stack pointer >>>> that my context switch will now preserve and: >>>> - determine if it is deeper into the stack than any previously recorded >>>> instance >>>> - if so, store that value (deepest_stack_pointer) and >>>> - verify that it is within the range of valid addresses for the memory >>>> allocated for the stack (if not, fall into the debugger before the >>>> "contamination" spreads, obfuscating the underlying *cause* >>>> Then, periodically, explore the region *beyond* the stack pointer >>>> to see how much of this "regular pattern" has been obliterated *between* >>>> reschedule()'s. >>> >>> Oy. I think you're explaining exactly why I prefer the way I do it :) >>> I'll also make sure that plenty of stack is allocated; over do it. >> >> I live with tightly constrained resources. It's important for me to size >> large objects (e.g., stacks, heaps, etc.) to fit their actual *needs* and >> not just "throw memory at them". E.g., my memory allocator lets me "trim" >> allocations (instead of being forced to release them in the same chunks >> that they were allocated) as well as *extend* existing allocations (the >> policy that the allocator uses can be specified in the allocation >> request -- e.g., find me a piece of memory that adjoins *this* piece). >> >> This allows me to "move" memory between a consumer and producer. E.g., >> arrange for the producer and consumer's memory to be contiguous and "free" >> it from one while "alloc"ing it to the other. > > Memory is cheap.
No, it isn't. I have several designs, currently, where memory is *finite*. Adding memory means moving to a bigger SoC *or* moving to a multi-chip solution (when the range of SoC choices with "sufficient memory" falls below a threshold that makes the functionality I need "unavailable"). These aren't like "PC-based" designs where you can replace a 1G DIMM with a 4G DIMM, reboot and be fat happy and glorious.
>>>>> I suppose the major thing driving me here is to use development >>>>> techniques >>>>> which allow me a better chance to find out about these potential >>>>> issues in >>>>> a controlled deterministic manner as early on in the development >>>>> process >>>>> as possible. >>>> >>>> The first time you have to chase down this sort of "problem" will pay >>>> for >>>> every precaution you ever take against it in future efforts! :-/ >>> >>> I haven't seen a stack overflow in decades, excepting where I'm porting >>> code. >> >> Then you haven't been tasked with trimming your stack to fit the needs of >> the routines using it! Or your heaps. :> > > Newp - haven't run into that in ages. It's just time-intensive. I'm sitting on > about 100K lines of code these days. Maybe more.
Code size doesn't matter. I can write 10K lines of code that never use more than 200 *bytes* of RAM (been there, done that). I can write 500 lines of code that will gobble up gigabytes and still crave more! (in neither case being "pathological") ROM is cheap. FLASH, less so. RAM, least of all. (then cache, registers, etc.) E.g., I can unroll a loop to save an iterator. I can inline function calls to save stack frames. etc. OTOH, once RAM is exhausted, I either come up with a new algorithm, move to a new platform *or* consider the problem as unsolvable given the space/power/cost constraints placed on it with today's choice of components.
>> One of the "problems" with "embedded" is it groups a wide variety of >> application domains and implementation constraints into a single subject. >> E.g., in the same application/codebase, I have devices that use Q10.14 >> while others use "Big Rationals" based on the resources available to each. > > I'm just saying - there are $50 ARM systems - COTS - that sport gigs of > memory these days.
Great! I'm designing an earpiece. How many gigs can I fit in that "ear canal"? *ASSUMING* cost is not an issue?? And, where will the user carry the battery for it all?? (in, perhaps, the *other* ear?? :> ) What if your price point is $9.95? (e.g., the price of many earpieces) As I said (immediately above) "embedded" encompasses a very wide range of targets -- price points, physical sizes, operating power, etc.
Hans-Bernhard Br&ouml;ker <HBBroeker@t-online.de> writes:
> Not really. Supposing you start out with a correct program and > compiler, making previously automatic variables static can never > reduce memory consumption. It can only increase it.
I think the idea was not necessarily to minimize memory consumption, but rather to ensure that memory consumption stayed within the available bound. E.g. if you have 8k of memory available and your program uses 7k, there's no real benefit in shrinking it to 6k. But if you're allocating buffers or other large objects on the stack, you have to do very careful accounting of the possible call trees, to know how much stack memory can be consumed by nested allocations. But if you allocate those buffers statically, the linker can tell you their total consumption without your having to analyze too much. That just leaves a few dozen bytes per call level for some temporaries, so it's mainly a matter of preventing excessively deep nesting such as from runaway recursion. It might be interesting to see MISRA C's recommendations for this.
On Sun, 14 Dec 2014 01:21:13 +0100, Hans-Bernhard Br&#4294967295;ker
<HBBroeker@t-online.de> wrote:

>Am 13.12.2014 um 21:22 schrieb Les Cargill: >> Hans-Bernhard Br&#4294967295;ker wrote: > >>> Making variables static when they don't need to be blows up memory consumption >>> considerably. > >> This varies. > >Not really. Supposing you start out with a correct program and >compiler, making previously automatic variables static can never reduce >memory consumption. It can only increase it.
I have been working with computers that did not even have any concept of stack. Many had some kind of page 0 addressing in which a few bits (typically 5-12) was used to access variables in low memory in page 0. That low memory access is similar to 680x direct addressing mode (1 byte opcode and 1 byte address to the low 256 memory locations). Of cause, these low memory locations can be reused as local variables in multiple functions, as long as they do not directly or indirectly call each other. I do not see why a static allocation would be any larger than stack allocation. In processors without stack pointer relative addressing modes, this will significantly reduce the code size. Typically, a compiler intended for such primitive processors usually have means of defining the load address of variables or structures into the address space, making it possible to allocate variables into the same addresses, at least in separately compiled modules.
"Don Y" <this@is.not.me.com> wrote in message 
news:m6i1dm$d2c$1@speranza.aioe.org...
> Hi Simon, > > On 12/13/2014 6:08 AM, Simon Clubley wrote: >> On 2014-12-13, Don Y <this@is.not.me.com> wrote: >>> >>> Braces really help sort out nesting on if-else. >> >> In my personal coding standards, _everything_ (ie: single statements) >> gets placed between braces in brace orientated languages. > > Agreed. It is too difficult to be tricked (e.g., by indent) into THINKING > braces exist where they don't. Especially if your coding style tries to > minimize newlines.
Please explain how much does a new line cost on your machine? tim
Am 14.12.2014 um 08:08 schrieb Paul Rubin:
> Hans-Bernhard Br&ouml;ker <HBBroeker@t-online.de> writes:
>> Not really. Supposing you start out with a correct program and >> compiler, making previously automatic variables static can never >> reduce memory consumption. It can only increase it.
> I think the idea was not necessarily to minimize memory consumption,
The idea I criticized was that making things static was a good strategy on memory-constrained systems. Which it's not.
> but rather to ensure that memory consumption stayed within the > available bound.
And it fails that goal by _increasing_ memory consumption, moving it beyond that available bound with considerable likelihood. I.e. it creates the very problem that it's supposed to help avoid.
> E.g. if you have 8k of memory available and your program uses > 7k, there's no real benefit in shrinking it to 6k.
Not until you need another 1.5 KiB, that is.
> But if you're allocating buffers or other large objects on the stack, > you have to do very careful accounting of the possible call trees, to > know how much stack memory can be consumed by nested allocations.
As I said before, you need to do that accounting anyway, so that's not really much of an argument either way. Compilers for small-ish embedded systems worth their salt usually come with a static stack analyzer for that purpose.
> But if you allocate those buffers statically, the linker can tell you > their total consumption without your having to analyze too much.
And it will tell you that your program failed, long before it actually has to.
> It might be interesting to see MISRA C's recommendations for this.
There are none.
The 2026 Embedded Online Conference