load binary modules at run time

Hi guys,

I have 100 c modules, my system has to run only one module at a time (at run time); these modules have few functions called by the system and these have the same prototipe for every module. These modules has a lot of dependencies with the system.
My first approach was create only one project with all files, assign one section for each module, specify that all these sections must run in the same section.

Example: module A, B

name=".text_A" runin=".text_runmodule"
name=".rodata_A" runin=".rodata_runmodule"
name=".data_A" runin=".data_runmodule"
name=".bss_A" runin=".bss_runmodule"

name=".text_B" runin=".text_runmodule"
name=".rodata_B" runin=".rodata_runmodule"
name=".data_B" runin=".data_runmodule"
name=".bss_B" runin=".bss_runmodule"


with objcopy I extracted all the sections so I can create A.bin a B.bin e load these files at run time on the runmodule sections (in RAM)

I created an array of a structure that contains pointer to the module's functions.

drawbacks are:
1 - I have to create a lot of sections
2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..

Any suggestion? Static or shared library? What else?..

Thanks,
Fabius

On 21.5.14 13:28, FabiusAone wrote:
>
> Hi guys,
>
> I have 100 c modules, my system has to run only one module at a time (at run time); these modules have few functions called by the system and these have the same prototipe for every module. These modules has a lot of dependencies with the system.
> My first approach was create only one project with all files, assign one section for each module, specify that all these sections must run in the same section.
>
> Example: module A, B
>
> name=".text_A" runin=".text_runmodule"
> name=".rodata_A" runin=".rodata_runmodule"
> name=".data_A" runin=".data_runmodule"
> name=".bss_A" runin=".bss_runmodule"
>
> name=".text_B" runin=".text_runmodule"
> name=".rodata_B" runin=".rodata_runmodule"
> name=".data_B" runin=".data_runmodule"
> name=".bss_B" runin=".bss_runmodule"
>
>
> with objcopy I extracted all the sections so I can create A.bin a B.bin e load these files at run time on the runmodule sections (in RAM)
>
> I created an array of a structure that contains pointer to the module's functions.
>
> drawbacks are:
> 1 - I have to create a lot of sections
> 2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
> 3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..
>
> Any suggestion? Static or shared library? What else?..
>
> Thanks,
> Fabius



This is the classical overlay structure. Look in the linker/loader
manual for 'Overlay loading'. Google may also help.

-- 

Tauno Voipio

In comp.arch.embedded,
FabiusAone <fabio.ferrari.aone@gmail.com> wrote:
>
> I have 100 c modules, my system has to run only one module at a time (at run time); these modules have few functions called by the system and these have the same prototipe for every module. These modules has a lot of dependencies with the system.

<snip>

> Any suggestion? Static or shared library? What else?..

My first thought was "overlays". Used mostly one or two decades ago, I think.
But may be a nice term to search for. ;-)

-- 
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail)

Do clones have navels?

On 21/05/14 12:28, FabiusAone wrote:
> 
> Hi guys,
> 
> I have 100 c modules, my system has to run only one module at a time (at run time); these modules have few functions called by the system and these have the same prototipe for every module. These modules has a lot of dependencies with the system.
> My first approach was create only one project with all files, assign one section for each module, specify that all these sections must run in the same section.
> 
> Example: module A, B
> 
> name=".text_A" runin=".text_runmodule"
> name=".rodata_A" runin=".rodata_runmodule"
> name=".data_A" runin=".data_runmodule"
> name=".bss_A" runin=".bss_runmodule"
> 
> name=".text_B" runin=".text_runmodule"
> name=".rodata_B" runin=".rodata_runmodule"
> name=".data_B" runin=".data_runmodule"
> name=".bss_B" runin=".bss_runmodule"
> 
> 
> with objcopy I extracted all the sections so I can create A.bin a B.bin e load these files at run time on the runmodule sections (in RAM)
> 
> I created an array of a structure that contains pointer to the module's functions.
> 
> drawbacks are:
> 1 - I have to create a lot of sections
> 2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
> 3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..
> 
> Any suggestion? Static or shared library? What else?..
> 
> Thanks,
> Fabius
> 

What sort of tools are you using?  "objcopy" implies gcc.  But "create
one project" implies that you are using an IDE to handle the build
process.  My thoughts here are that gcc, ln, objcopy, and make can
combine to produce magic, but you will have to learn the joys of make
and Makefiles.  It can be hard going, and debugging Makefiles is "fun",
but it should be possible to get a build system together so that when
you change a file, all dependent modules get rebuilt as necessary, but
nothing else.

There are a couple of shortcuts that can make a big difference for large
builds, whether you are using your own Makefile or an IDE.  First, make
sure you have parallel build enabled with an appropriate number of jobs
for the number of cores in your PC.  Second, try to use ccache to make
redundant compiles a little faster.


The other thing you should probably do is decouple your modules and your
base system.  Arrange for your modules to have an entry point (or entry
points) at a fixed address at the start of the module.  This entry point
may register other functions with the base system, depending on the
flexibility you need.  Functions from the base system that are needed by
the modules should be exported as an array of pointers, and the address
of that array should be passed to the module via its entry function.
Once you have this sort of system in place, the base system and the
modules can be compiled and linked independently - there will be no need
to update everything each time something changes in the base system.

Thanks but...this is what I did..or not?

What about drawbacks?:
1 - I have to create a lot of sections 
2 - when I'm working with a module, I have to exclude from build the other modules to speed up time 
3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules.. 


> 
> This is the classical overlay structure. Look in the linker/loader
> 
> manual for 'Overlay loading'. Google may also help.
> 
> 
> 
> -- 
> 
> 
> 
> Tauno Voipio

Hi David,
I use CrossWorks (gcc) with ARM Cortex M4, parallel compile are enabled but it would be nice to have a base system that don't change each time you modify a module..
Your "decoupling" approach works well regarding module's functions needed by the base system, in fact I think that I'll maintain this approach that I tried in these days, but regarding the needs of the modules there's a problem: low level dependencies..for example __float64_div..I have no control of these symbols..
probably the best solution could be to force duplication of these pieces of code in each module..but I'm not able to do it..



> What sort of tools are you using?  "objcopy" implies gcc.  But "create
> 
> one project" implies that you are using an IDE to handle the build
> 
> process.  My thoughts here are that gcc, ln, objcopy, and make can
> 
> combine to produce magic, but you will have to learn the joys of make
> 
> and Makefiles.  It can be hard going, and debugging Makefiles is "fun",
> 
> but it should be possible to get a build system together so that when
> 
> you change a file, all dependent modules get rebuilt as necessary, but
> 
> nothing else.
> 
> 
> 
> There are a couple of shortcuts that can make a big difference for large
> 
> builds, whether you are using your own Makefile or an IDE.  First, make
> 
> sure you have parallel build enabled with an appropriate number of jobs
> 
> for the number of cores in your PC.  Second, try to use ccache to make
> 
> redundant compiles a little faster.
> 
> 
> 
> 
> 
> The other thing you should probably do is decouple your modules and your
> 
> base system.  Arrange for your modules to have an entry point (or entry
> 
> points) at a fixed address at the start of the module.  This entry point
> 
> may register other functions with the base system, depending on the
> 
> flexibility you need.  Functions from the base system that are needed by
> 
> the modules should be exported as an array of pointers, and the address
> 
> of that array should be passed to the module via its entry function.
> 
> Once you have this sort of system in place, the base system and the
> 
> modules can be compiled and linked independently - there will be no need
> 
> to update everything each time something changes in the base system.

FabiusAone <fabio.ferrari.aone@gmail.com> writes:

> Hi guys,
>
> I have 100 c modules, my system has to run only one module at a time
> (at run time); these modules have few functions called by the system
> and these have the same prototipe for every module. These modules has
> a lot of dependencies with the system.
> My first approach was create only one project with all files, assign one section for each module, specify that all these sections must run in the same section.
>
> Example: module A, B
>
> name=".text_A" runin=".text_runmodule"
> name=".rodata_A" runin=".rodata_runmodule"
> name=".data_A" runin=".data_runmodule"
> name=".bss_A" runin=".bss_runmodule"
>
> name=".text_B" runin=".text_runmodule"
> name=".rodata_B" runin=".rodata_runmodule"
> name=".data_B" runin=".data_runmodule"
> name=".bss_B" runin=".bss_runmodule"
>
>
> with objcopy I extracted all the sections so I can create A.bin a B.bin e load these files at run time on the runmodule sections (in RAM)
>
> I created an array of a structure that contains pointer to the module's functions.
>
> drawbacks are:
> 1 - I have to create a lot of sections
> 2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
> 3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..
>
> Any suggestion? Static or shared library? What else?..
>
> Thanks,
> Fabius

What the hell are you trying to do? Even a smallish ARM Cortex M4 has
512 KB of FLASH these days. Why are you trying to overlay?!?!?
-- 
Randy Yates
Digital Signal Labs
http://www.digitalsignallabs.com

2014-05-21 12:28, FabiusAone skrev:
>
> Hi guys,
>
> I have 100 c modules, my system has to run only one module at a time (at run time);
 > these modules have few functions called by the system and these have 
the same prototipe
 > for every module. These modules has a lot of dependencies with the
 > system.
> My first approach was create only one project with all files, assign one section for
> each module, specify that all these sections must run in the same section.
>
> Example: module A, B
>
> name=".text_A" runin=".text_runmodule"
> name=".rodata_A" runin=".rodata_runmodule"
> name=".data_A" runin=".data_runmodule"
> name=".bss_A" runin=".bss_runmodule"
>
> name=".text_B" runin=".text_runmodule"
> name=".rodata_B" runin=".rodata_runmodule"
> name=".data_B" runin=".data_runmodule"
> name=".bss_B" runin=".bss_runmodule"
>
>
> with objcopy I extracted all the sections so I can create A.bin a B.bin e load these
> files at run time on the runmodule sections (in RAM)
>
> I created an array of a structure that contains pointer to the module's functions.
>
> drawbacks are:
> 1 - I have to create a lot of sections
> 2 - when I'm working with a module, I have to exclude from build the other modules
 > to speed up time
> 3 - every time I modify a module I have to recompile all the modules because system
   has changed..and theoretically I SHOULD retest all the modules..
>
> Any suggestion? Static or shared library? What else?..
>
> Thanks,
> Fabius
>

Why not map each module to a task ,and only run a single task at a time?

You want a message passing RTOS.

BR
Ulf Samuelsson

On 21/05/14 16:59, FabiusAone wrote:
>
> Hi David, I use CrossWorks (gcc) with ARM Cortex M4, parallel compile
> are enabled but it would be nice to have a base system that don't
> change each time you modify a module.. Your "decoupling" approach
> works well regarding module's functions needed by the base system, in
> fact I think that I'll maintain this approach that I tried in these
> days, but regarding the needs of the modules there's a problem: low
> level dependencies..for example __float64_div..I have no control of
> these symbols.. probably the best solution could be to force
> duplication of these pieces of code in each module..but I'm not able
> to do it..
>

(Please get yourself a newsreader and a newsserver, rather than the 
crappy Google Groups interface.  Google Groups is fine for searching, 
but screws up the formatting of posts.  Thunderbird and 
news.eternal-september.org are free and work on all platforms.  And 
please don't top-post.)

Make your modules separate projects, so that they are self-contained 
except for accessing specific functions in the base system via a jump 
table.  They can even have a "main" function - this can be the entry 
point for the module.  You will end up with duplication of library code 
between the base system and modules, but that is unlikely to matter - as 
long as you follow good practices of not using floating point 
(especially doubles) unless they are needed, not using things like 
printf(), etc., then the duplicated library code is going to be fairly 
small.  If there are some functions that you know you need and you know 
they cause a lot of extra code, such as printf(), you can put them in 
the jump table.  You can't necessarily do it with gcc library functions 
(as distinct from newlib or libc functions) - functions starting with 
two underscores are compiler-private functions and may have weird 
calling conventions that don't work when you try to use function 
pointers.  And if you use malloc() and free(), you have to keep them in 
the base system rather than having two copies.

David


>
>
>> What sort of tools are you using?  "objcopy" implies gcc.  But
>> "create
>>
>> one project" implies that you are using an IDE to handle the build
>>
>> process.  My thoughts here are that gcc, ln, objcopy, and make can
>>
>> combine to produce magic, but you will have to learn the joys of
>> make
>>
>> and Makefiles.  It can be hard going, and debugging Makefiles is
>> "fun",
>>
>> but it should be possible to get a build system together so that
>> when
>>
>> you change a file, all dependent modules get rebuilt as necessary,
>> but
>>
>> nothing else.
>>
>>
>>
>> There are a couple of shortcuts that can make a big difference for
>> large
>>
>> builds, whether you are using your own Makefile or an IDE.  First,
>> make
>>
>> sure you have parallel build enabled with an appropriate number of
>> jobs
>>
>> for the number of cores in your PC.  Second, try to use ccache to
>> make
>>
>> redundant compiles a little faster.
>>
>>
>>
>>
>>
>> The other thing you should probably do is decouple your modules and
>> your
>>
>> base system.  Arrange for your modules to have an entry point (or
>> entry
>>
>> points) at a fixed address at the start of the module.  This entry
>> point
>>
>> may register other functions with the base system, depending on
>> the
>>
>> flexibility you need.  Functions from the base system that are
>> needed by
>>
>> the modules should be exported as an array of pointers, and the
>> address
>>
>> of that array should be passed to the module via its entry
>> function.
>>
>> Once you have this sort of system in place, the base system and
>> the
>>
>> modules can be compiled and linked independently - there will be no
>> need
>>
>> to update everything each time something changes in the base
>> system.
>

>
> What the hell are you trying to do? Even a smallish ARM Cortex M4 has
> 512 KB of FLASH these days. Why are you trying to overlay?!?!?
>
5MB..for now..