Reply by D Yuniskis July 2, 20102010-07-02
Hi Peter,

Peter Dickerson wrote:

[8<]

>>>> Yes, we wrote/implemented a "QBASIC" for some of our instruments >>>> for just this reason (blood assays). Allowed the customer to >>>> design new tests without having to contract with us to code >>>> them. I.e., we just provided a device that came up with >>>> the raw data and let the customer come up with the means >>>> of interpreting that data based on the reagents, etc. that >>>> he was using in the assay. >>>> >>>> Note that you can do this two different ways: >>>> - *source* level interpreter in the instrument >>>> - "bytecode" interpreter in the instrument with >>>> an external "compiler/parser". >>> Yes, I'd go for source since that is conceptually the simplest. Otherwise >>> I need a bytecode compiler somewhere in the machine or on a PC. >> I should probably clarify... :< >> >> To us, customer was the OEM. *They* would design new >> assays, debug them (in terms of the chemistry) and >> code up the "QBASIC" routines to implement those assays >> for *their* customers ("end users"). So, it was reasonable >> to expect them to have certain tools available for that >> process (though they surely wouldn't want to be dealing >> with "raw iron"). > > Basically the same here except that when the customer snaps his fingers we > jump. Very few customers and all big. In one case we have something like 14 > versions of an instrument. Actually, more than one case, because often the > casework and badge is the the main difference. Part of the problem though is > that each "Method" or assay has a lots of UI parameters to set, and lots of > languages to work in (including Chinese and Japanese).
Ah. In our case, *one* customer (the instrument was designed to fit their needs). But, from their (and our!) perspective, fiddling around with chemistries and tweaking assays just didn't make sense for *us* to be doing. Their folks could tweak the reagents used in the assay, observe the results and create their own "test criteria" to present *their* results to *their* customers far more efficiently than if we were in the middle of the process. We made some special "ROM Paks" with BBSRAM (and a "write protect" switch) to facilitate their development efforts but otherwise stayed clear of it all. <shrug> I suspect it made sense economically (though I am sure the decision was made more from an *engineering* perspective -- being engineers, we didn't want to be "bothered" with the boring details of assays ;> ) [frankly, I wouldn't have wanted to be *around* some of the things (bugs!) they were testing for...]
Reply by Paul Keinanen July 2, 20102010-07-02
On Fri, 2 Jul 2010 00:16:55 -0700 (PDT), nono240 <nono240@gmail.com>
wrote:

>Thanks for all your replies ! > >> If you can run only one task at a time, why do you need position >> independent code ? Just link each program to the same fixed load >> address. You need PIC code only when there are _multiple_ programs to >> be loaded somewhere into the RAM. > > >Because those "tasklets" are stored in different places in FLASH : we >ship the device with 4 embedded tasklets, >but a dozen more are available to download and free to be uploaded at >any of those 4 slots, we don't want the user to take care about "link >address" ! >Moreover, if we update our CPU to more FLASH, we don't want to deal >with multiple tasklets version. > > >> If you want to share data between these programs, first link the data >> area to a fixed address and then link each program to that address. >> >> This is how it was done half a century ago. In FORTRAN, create a >> COMMON area, install it into a fixed address (usually at the top of >> the core) and then load each "transient program" into low memory, >> since the whole program could not fit into the core at once. No >> base/stack pointer relative addressing needed, since the data >> addresses were known at compile time. > >Relocating a tasklet "on demand" to an "predefined fixed area" will >prematurely kill the FLASH since there's not enough RAM to run code >from...
I was suggesting putting data _AND_ code into the 8 KiB RAM. Assuming 4 KiB for data and 4 KiB allocated for code, you could use functions up to 4 KiB in length. When a function is called, it is copied from a unique address in FLASH into RAM starting at address 0 and this function is then executed starting at 0. When this function calls something else that needs to be loaded, the current segment ID and offset from start must be saved, so that the calling function code can reload the code and continue execution after the call instruction. With the fast flash read times would make such systems much faster than the old overlay loading systems loading from slow disk drives.
> >> With modern processors with versatile addressing modes, why not >> reserve one data area pointer at a known location (such as the first >> or last address in RAM or ROM) and use this to access the shared >> variables in each program ? >> >> &#4294967295; Data = GetPersistentDataAreaPointer() ; >> &#4294967295; ... >> &#4294967295; Data->SharedVar1 = Data->SharedVar2 ; > >Because we want the tasklets to be "RTOS" unaware. Our FreeRTOS is >running as an "hypervisor" (we did have an MPU though). > >I just need a way to tell LD that my DATA section is ABSOLUTE, and not >relative from CODE..
Why do you need a DATA section ? Think about the system initialization routine as the "main" function and within that function, allocate all variables as automatic local variables that needs to be shared between the tasklets. The tasklets can then be called as subfunctions and each call will then pass pointers to the "main" local variables as call parameters. In order to keep the tasklet parameter list length reasonable, the "main" local variables are grouped into structures.
Reply by Peter Dickerson July 2, 20102010-07-02
"D Yuniskis" <not.going.to.be@seen.com> wrote in message 
news:i0kl5a$jvh$1@speranza.aioe.org...
> Hi Peter, > > Peter Dickerson wrote: >> "D Yuniskis" <not.going.to.be@seen.com> wrote in message >> news:i0kg59$cqf$1@speranza.aioe.org... >>> Hi Peter, >> [snip] >> >>> Yes, we wrote/implemented a "QBASIC" for some of our instruments >>> for just this reason (blood assays). Allowed the customer to >>> design new tests without having to contract with us to code >>> them. I.e., we just provided a device that came up with >>> the raw data and let the customer come up with the means >>> of interpreting that data based on the reagents, etc. that >>> he was using in the assay. >>> >>> Note that you can do this two different ways: >>> - *source* level interpreter in the instrument >>> - "bytecode" interpreter in the instrument with >>> an external "compiler/parser". >> >> Yes, I'd go for source since that is conceptually the simplest. Otherwise >> I need a bytecode compiler somewhere in the machine or on a PC. > > I should probably clarify... :< > > To us, customer was the OEM. *They* would design new > assays, debug them (in terms of the chemistry) and > code up the "QBASIC" routines to implement those assays > for *their* customers ("end users"). So, it was reasonable > to expect them to have certain tools available for that > process (though they surely wouldn't want to be dealing > with "raw iron").
Basically the same here except that when the customer snaps his fingers we jump. Very few customers and all big. In one case we have something like 14 versions of an instrument. Actually, more than one case, because often the casework and badge is the the main difference. Part of the problem though is that each "Method" or assay has a lots of UI parameters to set, and lots of languages to work in (including Chinese and Japanese).
> I don't think the end user ever *saw* QBASIC. They would > typically purchase "Paks" (ROM based) for each assay. > FDA issues, etc. (though there was no reason why, from > *our* standpoint, this utility couldn't be "exposed")
[snip]
Reply by D Yuniskis July 2, 20102010-07-02
Hi Peter,

Peter Dickerson wrote:
> "D Yuniskis" <not.going.to.be@seen.com> wrote in message > news:i0kg59$cqf$1@speranza.aioe.org... >> Hi Peter, > [snip] > >> Yes, we wrote/implemented a "QBASIC" for some of our instruments >> for just this reason (blood assays). Allowed the customer to >> design new tests without having to contract with us to code >> them. I.e., we just provided a device that came up with >> the raw data and let the customer come up with the means >> of interpreting that data based on the reagents, etc. that >> he was using in the assay. >> >> Note that you can do this two different ways: >> - *source* level interpreter in the instrument >> - "bytecode" interpreter in the instrument with >> an external "compiler/parser". > > Yes, I'd go for source since that is conceptually the simplest. Otherwise I > need a bytecode compiler somewhere in the machine or on a PC.
I should probably clarify... :< To us, customer was the OEM. *They* would design new assays, debug them (in terms of the chemistry) and code up the "QBASIC" routines to implement those assays for *their* customers ("end users"). So, it was reasonable to expect them to have certain tools available for that process (though they surely wouldn't want to be dealing with "raw iron"). I don't think the end user ever *saw* QBASIC. They would typically purchase "Paks" (ROM based) for each assay. FDA issues, etc. (though there was no reason why, from *our* standpoint, this utility couldn't be "exposed")
>> (I'm talking *really* limited resources, here) >> >> If you have a more fleshy implementation to work with, >> look at Lua. Lately I am doing a lot with Inferno/Limbo >> (but would not suggest it for "end users") > > I did get Lua linked in but ran out of memory almost immediately. In > particular I couldn't measure anything. The problem seems to be that a lot > of stuff gets stored in RAM - dictionaries, strings etc. I'd prefer to be > able to keep that stuff in Flash only even at the cost of performance.
Yes, I find that to be true of "modern" languages. :< They try to be overly friendly instead of overly *efficient*. Hence my comments regarding fixing/declaring string sizes, integer only math, etc. You can do a *lot* in an environment thusly constrained *if* you are made aware of those constraints. Often, a "user" only needs the ability to do calculations, handle conditionals and "print" (emit?) formatted text with "results". So, a lot of the flowery features of modern languages are wasted... Nowadays, I would look into support for connectivity as more and more devices talk to each other (or *should*! :> ) One advantage of Limbo/Inferno is the application can extend beyond the confines of the device itself. E.g., you can "export" the hardware from the device/instrument and actually run the *application* external to the device (remotely!)
Reply by Peter Dickerson July 2, 20102010-07-02
"D Yuniskis" <not.going.to.be@seen.com> wrote in message 
news:i0kg59$cqf$1@speranza.aioe.org...
> Hi Peter,
[snip]
> Yes, we wrote/implemented a "QBASIC" for some of our instruments > for just this reason (blood assays). Allowed the customer to > design new tests without having to contract with us to code > them. I.e., we just provided a device that came up with > the raw data and let the customer come up with the means > of interpreting that data based on the reagents, etc. that > he was using in the assay. > > Note that you can do this two different ways: > - *source* level interpreter in the instrument > - "bytecode" interpreter in the instrument with > an external "compiler/parser".
Yes, I'd go for source since that is conceptually the simplest. Otherwise I need a bytecode compiler somewhere in the machine or on a PC.
> (I'm talking *really* limited resources, here) > > If you have a more fleshy implementation to work with, > look at Lua. Lately I am doing a lot with Inferno/Limbo > (but would not suggest it for "end users")
I did get Lua linked in but ran out of memory almost immediately. In particular I couldn't measure anything. The problem seems to be that a lot of stuff gets stored in RAM - dictionaries, strings etc. I'd prefer to be able to keep that stuff in Flash only even at the cost of performance. Peter
Reply by D Yuniskis July 2, 20102010-07-02
Hi Peter,

Peter Dickerson wrote:
> "D Yuniskis" <not.going.to.be@seen.com> wrote in message > news:i0hm4l$c6m$1@speranza.aioe.org... >> Hi Peter, >> >> Peter Dickerson wrote: >>> "D Yuniskis" <not.going.to.be@seen.com> wrote in message >>> news:i0gf3s$kou$1@speranza.aioe.org... >>>> nono240 wrote: >>>> >>>>> My CPU has no MMU, very little RAM (8KB), and is running a modified >>>>> FreeRTOS. I'd like to have the ability to "load" and run some code >>>>> from USART/DATAFLASH to FLASH as a RTOS task. Of course, for >>>>> convenience, the compiled code must be fully position independent. >>>>> Using the -fPIC or -fpic option, looking at the assembler, the code >>>>> seems OK : a dynamically computed offset is applied to every >>>>> operations. >>>>> >>>>> BUT, looking deeply, both DATA and CODE are applied the base offset ! >>>>> While this is the expected behavior for CODE (running anywhere in >>>>> FLASH), moving my CODE over the entire flash space doesn't mean moving >>>>> my RAM ! This make only sense when executing everything from SDRAM ! >>>>> >>>>> I'm looking for a solution to generate position independent *code*, >>>>> but with position dependent *data* using GCC/LD... Any help ? >>>> I find, in resource starved applications, using interpreters >>>> is a big win. If you're loading apps dynamically, I suspect >>>> the speed penalty would be insignificant (esp with careful >>>> choice of language) >>> Any suggestions for such interpretters, Don? Experiences? >> Remember, this is c.a.e so, for the most part, you *know* >> what the application is -- and what it will *remain* >> (i.e., we're not looking at an environment where you have to >> be able to handle infinite variety of applications). >> >> In the past, I've written C-ish, PL/M-ish and BASIC-ish interpreters >> along with Forth. Note that you can use these as guidelines >> for a pseudo-language without strictly complying with any >> formal language definition. >> >> E.g., you can opt to implement integer only math instead of >> supporting "doubles", etc. You can force limits to be defined >> for string lengths (static memory allocation). You can >> discount recursion, etc. >> >> The advantage of interpreters has always seemed to be coming >> up with really tight representations of algorithms and >> spend "ROM" instead of needing space in (loadable) RAM... > > OK, different aim. In my case I have a scientific instrument that is making > various low level measurements. Users, who are typically chemists or > biochemists, want real answers not raw measurements. For this the apply > "Methods" that turn instrumental measurements into stuff like > concentrations. These methods are all pretty standard but there are lots of > them, with the occasional new one turning up. I'd prefer the applications > chemists to be able to implement the methods so that I can concentrate on > measuring femtoamps. So, I'm looking scriptable.
Yes, we wrote/implemented a "QBASIC" for some of our instruments for just this reason (blood assays). Allowed the customer to design new tests without having to contract with us to code them. I.e., we just provided a device that came up with the raw data and let the customer come up with the means of interpreting that data based on the reagents, etc. that he was using in the assay. Note that you can do this two different ways: - *source* level interpreter in the instrument - "bytecode" interpreter in the instrument with an external "compiler/parser". (I'm talking *really* limited resources, here) If you have a more fleshy implementation to work with, look at Lua. Lately I am doing a lot with Inferno/Limbo (but would not suggest it for "end users")
Reply by Peter Dickerson July 2, 20102010-07-02
"D Yuniskis" <not.going.to.be@seen.com> wrote in message 
news:i0hm4l$c6m$1@speranza.aioe.org...
> Hi Peter, > > Peter Dickerson wrote: >> "D Yuniskis" <not.going.to.be@seen.com> wrote in message >> news:i0gf3s$kou$1@speranza.aioe.org... >>> nono240 wrote: >>> >>>> My CPU has no MMU, very little RAM (8KB), and is running a modified >>>> FreeRTOS. I'd like to have the ability to "load" and run some code >>>> from USART/DATAFLASH to FLASH as a RTOS task. Of course, for >>>> convenience, the compiled code must be fully position independent. >>>> Using the -fPIC or -fpic option, looking at the assembler, the code >>>> seems OK : a dynamically computed offset is applied to every >>>> operations. >>>> >>>> BUT, looking deeply, both DATA and CODE are applied the base offset ! >>>> While this is the expected behavior for CODE (running anywhere in >>>> FLASH), moving my CODE over the entire flash space doesn't mean moving >>>> my RAM ! This make only sense when executing everything from SDRAM ! >>>> >>>> I'm looking for a solution to generate position independent *code*, >>>> but with position dependent *data* using GCC/LD... Any help ? >>> I find, in resource starved applications, using interpreters >>> is a big win. If you're loading apps dynamically, I suspect >>> the speed penalty would be insignificant (esp with careful >>> choice of language) >> >> Any suggestions for such interpretters, Don? Experiences? > > Remember, this is c.a.e so, for the most part, you *know* > what the application is -- and what it will *remain* > (i.e., we're not looking at an environment where you have to > be able to handle infinite variety of applications). > > In the past, I've written C-ish, PL/M-ish and BASIC-ish interpreters > along with Forth. Note that you can use these as guidelines > for a pseudo-language without strictly complying with any > formal language definition. > > E.g., you can opt to implement integer only math instead of > supporting "doubles", etc. You can force limits to be defined > for string lengths (static memory allocation). You can > discount recursion, etc. > > The advantage of interpreters has always seemed to be coming > up with really tight representations of algorithms and > spend "ROM" instead of needing space in (loadable) RAM...
OK, different aim. In my case I have a scientific instrument that is making various low level measurements. Users, who are typically chemists or biochemists, want real answers not raw measurements. For this the apply "Methods" that turn instrumental measurements into stuff like concentrations. These methods are all pretty standard but there are lots of them, with the occasional new one turning up. I'd prefer the applications chemists to be able to implement the methods so that I can concentrate on measuring femtoamps. So, I'm looking scriptable. Peter
Reply by Andrew Jackson July 2, 20102010-07-02
>> With modern processors with versatile addressing modes, why not >> reserve one data area pointer at a known location (such as the first >> or last address in RAM or ROM) and use this to access the shared >> variables in each program ? >> >> Data = GetPersistentDataAreaPointer() ; >> ... >> Data->SharedVar1 = Data->SharedVar2 ; > > Because we want the tasklets to be "RTOS" unaware. Our FreeRTOS is > running as an "hypervisor" (we did have an MPU though).
I don't see that Paul's suggestion makes your tasklet RTOS aware. Furthermore, I would have thought that the tasklets do need to be RTOS aware because they are manipulating a common data area.
> I just need a way to tell LD that my DATA section is ABSOLUTE, and not > relative from CODE..
What does your link script look like at present? Andrew
Reply by nono240 July 2, 20102010-07-02
Thanks for all your replies !

> If you can run only one task at a time, why do you need position > independent code ? Just link each program to the same fixed load > address. You need PIC code only when there are _multiple_ programs to > be loaded somewhere into the RAM.
Because those "tasklets" are stored in different places in FLASH : we ship the device with 4 embedded tasklets, but a dozen more are available to download and free to be uploaded at any of those 4 slots, we don't want the user to take care about "link address" ! Moreover, if we update our CPU to more FLASH, we don't want to deal with multiple tasklets version.
> If you want to share data between these programs, first link the data > area to a fixed address and then link each program to that address. > > This is how it was done half a century ago. In FORTRAN, create a > COMMON area, install it into a fixed address (usually at the top of > the core) and then load each "transient program" into low memory, > since the whole program could not fit into the core at once. No > base/stack pointer relative addressing needed, since the data > addresses were known at compile time.
Relocating a tasklet "on demand" to an "predefined fixed area" will prematurely kill the FLASH since there's not enough RAM to run code from...
> With modern processors with versatile addressing modes, why not > reserve one data area pointer at a known location (such as the first > or last address in RAM or ROM) and use this to access the shared > variables in each program ? > > =A0 Data =3D GetPersistentDataAreaPointer() ; > =A0 ... > =A0 Data->SharedVar1 =3D Data->SharedVar2 ;
Because we want the tasklets to be "RTOS" unaware. Our FreeRTOS is running as an "hypervisor" (we did have an MPU though). I just need a way to tell LD that my DATA section is ABSOLUTE, and not relative from CODE..
Reply by Albert van der Horst July 1, 20102010-07-01
In article <24ad21f6-9326-4fe8-8434-164dd88fdfd8@b35g2000yqi.googlegroups.com>,
nono240  <nono240@gmail.com> wrote:
>Hi there ! > >My CPU has no MMU, very little RAM (8KB), and is running a modified >FreeRTOS. I'd like to have the ability to "load" and run some code >from USART/DATAFLASH to FLASH as a RTOS task. Of course, for >convenience, the compiled code must be fully position independent. >Using the -fPIC or -fpic option, looking at the assembler, the code >seems OK : a dynamically computed offset is applied to every >operations. > >BUT, looking deeply, both DATA and CODE are applied the base offset ! >While this is the expected behavior for CODE (running anywhere in >FLASH), moving my CODE over the entire flash space doesn't mean moving >my RAM ! This make only sense when executing everything from SDRAM ! > >I'm looking for a solution to generate position independent *code*, >but with position dependent *data* using GCC/LD... Any help ?
A basic concept in linking ( ld program) is the ``section''. A section is an area of memory belonging together, such that e.g. distances within the section are fixable. A section may have a relocation table identifying the places in the section that still needs to be adjusted to the final place it will be used in the program. Now you want to have different sections behave differently regards location. What the linker (ld) does is combine sections from different object modules together into larger sections with names like .bss .text .data and possible fixing the relocation table. From that point whatever went into such a section will be treated in the same way, i.e. once you combined DATA and CODE into one section, data and code will be either fixed at a position or have a relocation table. The linker is blind to the difference between code and data, the only information it gets is by naming convention of input sections. This information is generated by the compiler. Now you have to understand which sections you have, then tell the linker what to do with it. Using the --debug option to the linker you get a so called linker script which details what the linker does. What you want can be accomplished by adapting the linker script, which is -- I admit -- not necessarily easy. Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst