On Mon, 07 Nov 2016 18:39:49 +0100, Wouter van Ooijen wrote:> Op 07-Nov-16 om 5:15 PM schreef Tim Wescott: >> On Mon, 07 Nov 2016 04:13:31 -0800, Dave Nadler wrote: >> >>> On Sunday, November 6, 2016 at 9:55:33 PM UTC-5, >>> anti...@math.uni.wroc.pl wrote: >>>> Dave Nadler <drn@nadler.com> wrote: >>>>> For years *one* of the *many* ways we've made reliable C++ >>>>> embedded is by forbidding dynamic storage (except heap). >>>> ^^^^ >>>> You mean stack? >>> >>> Sorry, stack only (no malloc/free/new/delete). >>> And of course static ctors. >>> >>> I was hoping for a standard way to: >>> - prune all memory allocation functions from libraries - stub any RTL >>> bits as you mention >>> >>> Thanks, >>> Best Regards,Dave >> >> I don't know if there's a standard way, but there are various ways. >> It's been a while since I've done it, but search on topics like >> "porting newlib to a bace-metal platform" or some such. Newlib is the >> run-time library for no-OS or RTOS operation, and it leaves about a >> dozen functions un- functioned. >> >> Personally, I allow for one-time object creation off of the heap, as >> long as it's all before the tasks start up and as long as nothing gets >> deleted and recreated. But that makes it hard to be strict about not >> allowing it. >> >> IIRC, newlib calls the "OS base memory allocation" procedure -- I don't >> think you can get rid of it entirely, but you can stub this off to an >> assert. If you _do_ get rid of all allocation off of the heap, then >> just leaving it out of your collection -o- stubs will make the link >> fail when someone does something that calls 'new'. > > That's exactly what I do: > > // using malloc must cause a linker error void * malloc( size_t n ){ > void if_you_see_this_you_are_using_new_or_malloc(); > if_you_see_this_you_are_using_new_or_malloc(); return NULL; > } > > // using free must cause a linker error void free( void * p ){ > void if_you_see_this_you_are_using_free_or_delete(); > if_you_see_this_you_are_using_free_or_delete(); > } > > And btw I hate global ctor's (ctor ordering problem), so my linkerscript > has: > > .IF_YOU_SEE_THIS_YOU_HAVE_ONE_OR_MORE_GLOBAL_OBJECT_CONSTRUCTORS : > { > KEEP(*(.init)); KEEP(*(.preinit_array)); KEEP(*(SORT(.init_array.*)));> KEEP(*(.init_array)); > } > nul > > Wouter "Objects? No Thanks!" van OoijenIf you're not going to allow global constructors, what's the point of using C++? As long as you're careful, and you can trust your initialization code to zero out BSS (which I can, because I wrote it, smirk smirk), then global constructor ordering issues are minor. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com I'm looking for work -- see my website!
g++ on Cortex-M with no dynamic memory
Started by ●November 5, 2016
Reply by ●November 7, 20162016-11-07
Reply by ●November 7, 20162016-11-07
Op 07-Nov-16 om 6:59 PM schreef Tim Wescott:> On Mon, 07 Nov 2016 18:39:49 +0100, Wouter van Ooijen wrote: > >> Op 07-Nov-16 om 5:15 PM schreef Tim Wescott: >>> On Mon, 07 Nov 2016 04:13:31 -0800, Dave Nadler wrote: >>> >>>> On Sunday, November 6, 2016 at 9:55:33 PM UTC-5, >>>> anti...@math.uni.wroc.pl wrote: >>>>> Dave Nadler <drn@nadler.com> wrote: >>>>>> For years *one* of the *many* ways we've made reliable C++ >>>>>> embedded is by forbidding dynamic storage (except heap). >>>>> ^^^^ >>>>> You mean stack? >>>> >>>> Sorry, stack only (no malloc/free/new/delete). >>>> And of course static ctors. >>>> >>>> I was hoping for a standard way to: >>>> - prune all memory allocation functions from libraries - stub any RTL >>>> bits as you mention >>>> >>>> Thanks, >>>> Best Regards,Dave >>> >>> I don't know if there's a standard way, but there are various ways. >>> It's been a while since I've done it, but search on topics like >>> "porting newlib to a bace-metal platform" or some such. Newlib is the >>> run-time library for no-OS or RTOS operation, and it leaves about a >>> dozen functions un- functioned. >>> >>> Personally, I allow for one-time object creation off of the heap, as >>> long as it's all before the tasks start up and as long as nothing gets >>> deleted and recreated. But that makes it hard to be strict about not >>> allowing it. >>> >>> IIRC, newlib calls the "OS base memory allocation" procedure -- I don't >>> think you can get rid of it entirely, but you can stub this off to an >>> assert. If you _do_ get rid of all allocation off of the heap, then >>> just leaving it out of your collection -o- stubs will make the link >>> fail when someone does something that calls 'new'. >> >> That's exactly what I do: >> >> // using malloc must cause a linker error void * malloc( size_t n ){ >> void if_you_see_this_you_are_using_new_or_malloc(); >> if_you_see_this_you_are_using_new_or_malloc(); return NULL; >> } >> >> // using free must cause a linker error void free( void * p ){ >> void if_you_see_this_you_are_using_free_or_delete(); >> if_you_see_this_you_are_using_free_or_delete(); >> } >> >> And btw I hate global ctor's (ctor ordering problem), so my linkerscript >> has: >> >> .IF_YOU_SEE_THIS_YOU_HAVE_ONE_OR_MORE_GLOBAL_OBJECT_CONSTRUCTORS : >> { >> KEEP(*(.init)); KEEP(*(.preinit_array)); KEEP(*(SORT > (.init_array.*))); >> KEEP(*(.init_array)); >> } > nul >> >> Wouter "Objects? No Thanks!" van Ooijen > > If you're not going to allow global constructors, what's the point of > using C++?If I had to mention just one feature: templates. And no, my templates don't bloat the generated code, they reduce the code size. And with constexpr the there is little need for global ctors. IMO global mutable objects are (genrally) a bad thing. Wouter "Objects? No Thanks!" van Ooijen
Reply by ●November 7, 20162016-11-07
On Mon, 7 Nov 2016 02:55:30 +0000 (UTC), antispam@math.uni.wroc.pl wrote:>Dave Nadler <drn@nadler.com> wrote: >> For years *one* of the *many* ways we've made reliable C++ >> embedded is by forbidding dynamic storage (except heap). > ^^^^ >You mean stack? Allocating memory at startup is reasonably >safe: each time you allocate the same amount so can easily >test if you stay within limit. Such allocation can be >useful for static constructors in library functions.Slightly off topic, why are C-programs so keen of using heap (malloc/free) instead of allocating temporary table variables (malloca) on stack as in e.g. Pascal ? Even some FORTRAN IV compilers supported something like SUBROUTINE FOO (N,M) DIMENSION TEMP (N,M) In which the floating point two dimensional TEMP array on the subroutine stack had different size, depending on which parameters were used to call FOO. Did this have something to do with the situation that some of the early processors used in C implementation did not support stack (or index register) relative addressing and only supported some kind of return address stack ?
Reply by ●November 7, 20162016-11-07
On Mon, 7 Nov 2016 18:39:49 +0100, Wouter van Ooijen <wouter@voti.nl> wrote:>Op 07-Nov-16 om 5:15 PM schreef Tim Wescott: >> On Mon, 07 Nov 2016 04:13:31 -0800, Dave Nadler wrote: >> >>> On Sunday, November 6, 2016 at 9:55:33 PM UTC-5, >>> anti...@math.uni.wroc.pl wrote: >>>> Dave Nadler <drn@nadler.com> wrote: >>>>> For years *one* of the *many* ways we've made reliable C++ >>>>> embedded is by forbidding dynamic storage (except heap). >>>> ^^^^ >>>> You mean stack? >>> >>> Sorry, stack only (no malloc/free/new/delete). >>> And of course static ctors. >>> >>> I was hoping for a standard way to: >>> - prune all memory allocation functions from libraries - stub any RTL >>> bits as you mention >>> >>> Thanks, >>> Best Regards,Dave >> >> I don't know if there's a standard way, but there are various ways. It's >> been a while since I've done it, but search on topics like "porting newlib >> to a bace-metal platform" or some such. Newlib is the run-time library >> for no-OS or RTOS operation, and it leaves about a dozen functions un- >> functioned. >> >> Personally, I allow for one-time object creation off of the heap, as long >> as it's all before the tasks start up and as long as nothing gets deleted >> and recreated. But that makes it hard to be strict about not allowing it. >> >> IIRC, newlib calls the "OS base memory allocation" procedure -- I don't >> think you can get rid of it entirely, but you can stub this off to an >> assert. If you _do_ get rid of all allocation off of the heap, then just >> leaving it out of your collection -o- stubs will make the link fail when >> someone does something that calls 'new'. > >That's exactly what I do: > >// using malloc must cause a linker error >void * malloc( size_t n ){ > void if_you_see_this_you_are_using_new_or_malloc(); > if_you_see_this_you_are_using_new_or_malloc(); > return NULL; >}What is wrong with malloc(), I use it during the first millisecond of the program startup.> >// using free must cause a linker error >void free( void * p ){ > void if_you_see_this_you_are_using_free_or_delete(); > if_you_see_this_you_are_using_free_or_delete(); >}free() is a no-no, I disable it after the first millisecond and let the program run for 10-40 years.> >And btw I hate global ctor's (ctor ordering problem), so my linkerscript >has: > >.IF_YOU_SEE_THIS_YOU_HAVE_ONE_OR_MORE_GLOBAL_OBJECT_CONSTRUCTORS : > { > KEEP(*(.init)); > KEEP(*(.preinit_array)); > KEEP(*(SORT(.init_array.*))); > KEEP(*(.init_array)); > } > nul > >Wouter "Objects? No Thanks!" van Ooijen > >
Reply by ●November 7, 20162016-11-07
On 11/7/2016 1:56 PM, upsidedown@downunder.com wrote:> On Mon, 7 Nov 2016 02:55:30 +0000 (UTC), antispam@math.uni.wroc.pl > wrote: > >> Dave Nadler <drn@nadler.com> wrote: >>> For years *one* of the *many* ways we've made reliable C++ >>> embedded is by forbidding dynamic storage (except heap). >> ^^^^ >> You mean stack? Allocating memory at startup is reasonably >> safe: each time you allocate the same amount so can easily >> test if you stay within limit. Such allocation can be >> useful for static constructors in library functions. > > Slightly off topic, why are C-programs so keen of using heap > (malloc/free) instead of allocating temporary table variables > (malloca) on stack as in e.g. Pascal ? > > Even some FORTRAN IV compilers supported something like > > SUBROUTINE FOO (N,M) > DIMENSION TEMP (N,M) > > In which the floating point two dimensional TEMP array on the > subroutine stack had different size, depending on which parameters > were used to call FOO. > > Did this have something to do with the situation that some of the > early processors used in C implementation did not support stack (or > index register) relative addressing and only supported some kind of > return address stack ?IIRC, variable length *auto* arrays were not supported prior to C99. There is also the issue of whether the array's size is part of its *type* -- or *value*. (and, in the latter case, if it is accessible as such!)
Reply by ●November 7, 20162016-11-07
On 11/7/2016 2:01 PM, upsidedown@downunder.com wrote:>> That's exactly what I do: >> >> // using malloc must cause a linker error >> void * malloc( size_t n ){ >> void if_you_see_this_you_are_using_new_or_malloc(); >> if_you_see_this_you_are_using_new_or_malloc(); >> return NULL; >> } > > What is wrong with malloc(), I use it during the first millisecond of > the program startup.If that's the only time you use it, you can *probably* replace each instance with a suitable variable/array declaration (global)>> // using free must cause a linker error >> void free( void * p ){ >> void if_you_see_this_you_are_using_free_or_delete(); >> if_you_see_this_you_are_using_free_or_delete(); >> } > > free() is a no-no, I disable it after the first millisecond and let > the program run for 10-40 years.So, all of your objects are persistent? (e.g., no "temporary/transient" tasks that come and go -- instantiating then releasing their stacks, etc.) How would you, for example, build a set of cascaded menus? In artificially introduced function layers?
Reply by ●November 7, 20162016-11-07
Op 07-Nov-16 om 10:01 PM schreef upsidedown@downunder.com:> On Mon, 7 Nov 2016 18:39:49 +0100, Wouter van Ooijen <wouter@voti.nl> > wrote: > >> Op 07-Nov-16 om 5:15 PM schreef Tim Wescott: >>> On Mon, 07 Nov 2016 04:13:31 -0800, Dave Nadler wrote: >>> >>>> On Sunday, November 6, 2016 at 9:55:33 PM UTC-5, >>>> anti...@math.uni.wroc.pl wrote: >>>>> Dave Nadler <drn@nadler.com> wrote: >>>>>> For years *one* of the *many* ways we've made reliable C++ >>>>>> embedded is by forbidding dynamic storage (except heap). >>>>> ^^^^ >>>>> You mean stack? >>>> >>>> Sorry, stack only (no malloc/free/new/delete). >>>> And of course static ctors. >>>> >>>> I was hoping for a standard way to: >>>> - prune all memory allocation functions from libraries - stub any RTL >>>> bits as you mention >>>> >>>> Thanks, >>>> Best Regards,Dave >>> >>> I don't know if there's a standard way, but there are various ways. It's >>> been a while since I've done it, but search on topics like "porting newlib >>> to a bace-metal platform" or some such. Newlib is the run-time library >>> for no-OS or RTOS operation, and it leaves about a dozen functions un- >>> functioned. >>> >>> Personally, I allow for one-time object creation off of the heap, as long >>> as it's all before the tasks start up and as long as nothing gets deleted >>> and recreated. But that makes it hard to be strict about not allowing it. >>> >>> IIRC, newlib calls the "OS base memory allocation" procedure -- I don't >>> think you can get rid of it entirely, but you can stub this off to an >>> assert. If you _do_ get rid of all allocation off of the heap, then just >>> leaving it out of your collection -o- stubs will make the link fail when >>> someone does something that calls 'new'. >> >> That's exactly what I do: >> >> // using malloc must cause a linker error >> void * malloc( size_t n ){ >> void if_you_see_this_you_are_using_new_or_malloc(); >> if_you_see_this_you_are_using_new_or_malloc(); >> return NULL; >> } > > What is wrong with malloc(), I use it during the first millisecond of > the program startup.If you need it, use it. I have an alternative version that I prefer not to use that supports malloc but not free. You can guess that its implementation is trivial. But if possible, I prefer not to use malloc at all, because it proves beyond doubt that - malloc isn't used after that initial phase - apart from stack size(s), the application will fit in memory This approach isn't suitable for all applications, but when it is suitable it a joy to use. Wouter "Objects? No Thanks!" van Ooijen
Reply by ●November 7, 20162016-11-07
On Mon, 07 Nov 2016 10:15:10 -0600, Tim Wescott wrote:> On Mon, 07 Nov 2016 04:13:31 -0800, Dave Nadler wrote: > >> On Sunday, November 6, 2016 at 9:55:33 PM UTC-5, >> anti...@math.uni.wroc.pl wrote: >>> Dave Nadler <drn@nadler.com> wrote: >>> > For years *one* of the *many* ways we've made reliable C++ >>> > embedded is by forbidding dynamic storage (except heap). >>> ^^^^ >>> You mean stack? >> >> Sorry, stack only (no malloc/free/new/delete). >> And of course static ctors. >> >> I was hoping for a standard way to: >> - prune all memory allocation functions from libraries - stub any RTL >> bits as you mention >> >> Thanks, >> Best Regards,Dave > > I don't know if there's a standard way, but there are various ways. > It's been a while since I've done it, but search on topics like "porting > newlib to a bace-metal platform" or some such. Newlib is the run-time > library for no-OS or RTOS operation, and it leaves about a dozen > functions un- functioned. > > Personally, I allow for one-time object creation off of the heap, as > long as it's all before the tasks start up and as long as nothing gets > deleted and recreated. But that makes it hard to be strict about not > allowing it. > > IIRC, newlib calls the "OS base memory allocation" procedure -- I don't > think you can get rid of it entirely, but you can stub this off to an > assert. If you _do_ get rid of all allocation off of the heap, then > just leaving it out of your collection -o- stubs will make the link fail > when someone does something that calls 'new'._sbrk is the function I was thinking of. I have a functioning one for the reasons given above. But, if you're going to do your own board- support package for newlib, there's about a dozen others that you need to address. -- Tim Wescott Control systems, embedded software and circuit design I'm looking for work! See my website if you're interested http://www.wescottdesign.com
Reply by ●November 8, 20162016-11-08
On Mon, 7 Nov 2016 14:14:50 -0700, Don Y <blockedofcourse@foo.invalid> wrote:>On 11/7/2016 2:01 PM, upsidedown@downunder.com wrote: >>> That's exactly what I do: >>> >>> // using malloc must cause a linker error >>> void * malloc( size_t n ){ >>> void if_you_see_this_you_are_using_new_or_malloc(); >>> if_you_see_this_you_are_using_new_or_malloc(); >>> return NULL; >>> } >> >> What is wrong with malloc(), I use it during the first millisecond of >> the program startup. > >If that's the only time you use it, you can *probably* replace each >instance with a suitable variable/array declaration (global)And recompile the source code for each application and each customer with different parameters. No thanks.> >>> // using free must cause a linker error >>> void free( void * p ){ >>> void if_you_see_this_you_are_using_free_or_delete(); >>> if_you_see_this_you_are_using_free_or_delete(); >>> } >> >> free() is a no-no, I disable it after the first millisecond and let >> the program run for 10-40 years. > >So, all of your objects are persistent? (e.g., no "temporary/transient" >tasks that come and go -- instantiating then releasing their stacks, etc.)The nice thing about stacks is that the worst case stack allocation can be determined statically. Even if recursion is used, the worst case allocation as long as you have a strict limit to the recursion depth. The problem with heaps is the fragmentation of the heap in a long running system. It might seem running OK for the first year, but will it run for the next ten years ? One can not test a system for 10 years and then release it. With properly designed systems, running it for a few months should be enough. If it is going to fail in the far future, it will fail during the early test period (infant mortality).>How would you, for example, build a set of cascaded menus? In >artificially introduced function layers?What prevents using a stack based allocation system ?
Reply by ●November 8, 20162016-11-08
On 11/8/2016 1:57 AM, upsidedown@downunder.com wrote:>>>> // using free must cause a linker error >>>> void free( void * p ){ >>>> void if_you_see_this_you_are_using_free_or_delete(); >>>> if_you_see_this_you_are_using_free_or_delete(); >>>> } >>> >>> free() is a no-no, I disable it after the first millisecond and let >>> the program run for 10-40 years. >> >> So, all of your objects are persistent? (e.g., no "temporary/transient" >> tasks that come and go -- instantiating then releasing their stacks, etc.) > > The nice thing about stacks is that the worst case stack allocation > can be determined statically. Even if recursion is used, the worst > case allocation as long as you have a strict limit to the recursion > depth. > > The problem with heaps is the fragmentation of the heap in a long > running system. It might seem running OK for the first year, but will > it run for the next ten years ?That depends on how that memory is physically implemented. E.g., slip a VMM under it and issues change (albeit at the granularity of pages) Also, it assumes your memory allocation strategy is fixed and not under the control of the memory system's clients -- who *know* how the memory is likely to be used (allocated AND released). [IMO, that's the problem with most allocators: one arena and one black-box allocation strategy.] E.g., folks have no problem dealing with "buffer pools" ("partitions", etc.) -- and PREFER them to "dynamic memory allocation". But, isn't this just a special case of dynamic memory allocation? One chosen where all requests are of a size determined a priori? If, for example, I implemented malloc() to allocate pools in exactly those same sizes, I could replace your "buffer_alloc()" with "malloc()" and your code would perform identically!> One can not test a system for 10 years > and then release it. With properly designed systems, running it for a > few months should be enough. If it is going to fail in the far future, > it will fail during the early test period (infant mortality).If I allocate task stacks from an arena that I, BY CONVENTION, manage as a set of "stack sized units" -- or, that is backed by a set of VMM pages that correspond efficiently with submultiples of stack sizes -- then I can allocate and free in damn near any order for all eternity without ever worrying about fragmentation. [It's just "buffer pools" at a different level in the implementation!] "Don't run with scissors" is unnecessarily pessimistic.>> How would you, for example, build a set of cascaded menus? In >> artificially introduced function layers? > > What prevents using a stack based allocation system ?Why force them to be auto variables in (layered) *function* stack frames when you can use an (intrinsic) stacking allocation strategy on a heap? (because that is the very essence of the user interface!) I.e., there are characteristics of the application of which the DEVELOPER is aware -- that a compiler nor stdlib writer will *never* be aware. Why tie the developers' hands needlessly? Are the standard libraries THAT intimidating that you can't "look under the hood" and FIX them to behave in a more opportune manner? To provide guarantees that SOME PARTICULAR IMPLEMENTATION doesn't? [Preface each function's name with "my_" if you're afraid someone won't understand that there are differences]
