Software Reuse In Embedded code

On 06/15/2011 01:25 PM, David Brown wrote:
> On 15/06/11 21:39, Tim Wescott wrote:
>> On 06/15/2011 08:20 AM, David Brown wrote:
>>> On 15/06/2011 16:03, steve wrote:
>>>> What percentage would you say your current project consists of
>>>> software reuseable items?
>>>>
>>>> By software reusable items I mean something that you acquired
>>>> (purchased, freeware, shareware, licensed etc) which you had no
>>>> responsibility in the development, construction, or testing. You
>>>> reused them because the documentation was sufficient to convince you
>>>> of its quality.
>>>
>>> 0 percent.
>>>
>>> If you use software from a third party, then /you/ are responsible for
>>> testing and otherwise qualifying it for use in /your/ system.
>>>
>>> Getting the software parts from somewhere you consider reliable will
>>> certainly /reduce/ the level of testing you need to do to be happy with
>>> its quality, but it does not eliminate it.
>>>
>>> If you buy your chips from a reliable supplier, and they come with good
>>> datasheets qualifying temperature ranges, power requirements, etc., does
>>> that mean you don't have to test your boards when you use them? It might
>>> mean you don't have to test over a wide temperature or voltage range,
>>> but you still have to test the boards.
>>>
>>> The same thing applies to software components.
>>
>> You write your libraries from scratch, or do all your work in assembly?
>>
>
> I don't use much third-party code (for small systems - the balance is
> different for larger systems). Obviously there is some library code with
> the compiler (especially language support code), though I don't tend to
> make much use of the standard C library.
>
> But my point is not that I or anyone else doesn't use third-party code -
> just that if I use third-party code in my project, then /I/ have a
> responsibility for testing, and making sure it is suitable for the job.
> The OP seems to think it is appropriate to take other people's code and
> use it, and let /them/ be responsible for any failures.
>
> Obviously the amount of testing and qualification you do with the
> third-party code depends on its source, and the type and quality of the
> code. You can generally rely on your compiler's integer division
> routines to give the correct result - but you would want to be a lot
> more careful about some code you found lying around on the internet or
> bought from a vendor you are not sure about (price is no indication of
> quality).

I was only yanking your chain about your "0 percent" -- the rest of it I 
agree with, wholeheartedly.

Even something as supposedly intrinsic as floating point code can't be 
trusted -- the early Code Composter library for the TMS320F28xx 
processor had a bug where numbers that were very close to an integer 
power of 2 would get doubled (or halved, I can't remember) when rounding 
to an integer, so 0.999999999 would end up as 2.0 (or 0.5), 254.99999999 
would end up as 512 (or 128), etc.

It was a royal pain in the behind, and fortunately they fixed it.  But 
the only comment I can add to your "you can't trust nothin'" is "that's 
right -- and you can't trust anything else, either".

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

Hi Tim,

On 6/15/2011 12:39 PM, Tim Wescott wrote:

> You write your libraries from scratch, or do all your work in assembly?

[disclaimer:  I write for a wide variety of application domains,
target hardware, etc. so my project-to-project needs vary
considerably.  If all of your projects "are the same", then
you're experiences will be different]

No to the assembly part but yes to the library query.

First, I tend to write my embedded projects *strictly* in C
(though I often use some esoteric "blend-ins").  And, most
tend to be "trimmed to fit" the resources available -- not
likely to have much of *anything* "to spare".

If you're developing in C++, my condolences -- I've yet to meet
the guy who can look at N pages of C++ code, identify all the
anonymous objects, every (invisible) constructor/destructor
instantiations, etc. and tell me, authoritatively, what that
code will behave like at run-time.  I usually can't afford
to have "surplus stack/heap/cpu" to cover the "unknowns".

Looking just at the C libraries, consider:

- do you use the float/double support in printf? scanf?  Or,
their support for long arguments?  Or pointers?  Take a peek at
how much bloat is associated with their presence in those
functions.  For each format that you *don't* need, does your
bastardized printf/scanf check for errors in the format string?
(or, do you just let that creep into your product as a "bug"?)

- what does your I/O system look like?  Where does printf()
send its output?  A UART? text display? bitmapped display via
a font rendering library?  Likewise, where does getchar()
fetch its input from?  Do you even *have* a "keyboard" -- or
is user input from a set of dedicated switches?

- do "stdout" and "stdin" really exist as entities?  Can you
redirect them to other physical devices?  Is fprintf(stdout...)
really the same as printf()?  Or, is everything hardwaired
(in practice) with lip service given to the mnemonics?

- do FILE*'s (and related functions) make sense in your world?
Can you *share* them among tasks (locking, etc.)?  Or, is the
behavior "undefined" when competing tasks access them?

- how do you implement resource locks?  Is your OS API always
consistent (so that the "lock_device(foo)" statement in the
stdio library behaves the same across platforms)?  Do your
tasks *share* a heap (which implies some sort of locking)?
Or, do you allocate a heap per task?

- ditto re: errno.  Or, has some "standard" forbidden you from
ever identifying the cause of your errors?  :>  What happens
if another task invokes a function that alters errno?

- is there any concept of "system()" in your environment?
Is there an *environment* in your environment?  :>  Who
catches exit()'s result?  etc.

- are you working in a flat memory model?  is there an
underlying VMM system?  are there *holes* in the memory
map (hint:  "yes")?  How do your libraries deal with these
issues -- have you come up with an implementation that
fits universally?  Or, do you just let the deviations
from one environment to the next result in *bugs*?

- do things like memmove() and memset() know how to take
advantage of the capabilities of the particular processor
(or, do they unconditionally resort to byte operations)?

- do you allow the str and mem functions to operate on
pointers that reference "read only" memory?  E.g., is it a
bug to strcpy to ROM?  What about *from* ROM?  What about
accessing arguments that reside in a stack frame or on the
heap... can one of these functions walk "past the end of
physical memory"?

Unless you've thought about how each function interacts with
your environment (neglecting the fact that the environment
might change from project-to-project), then you are leaving
your code vulnerable to more post-release bugs -- especially
if you don't have a thorough test plan in place (which I suspect
is the case with many projects).

The libraries that I release with a project are tailored to
*that* project.  They know the requirements and *assumptions*
that are appropriate for that project and live within those
confines (often deliberately exploiting certain characteristics
of those environments to make the code faster/simpler).

That's not to say the code doesn't *resemble* its counterparts
in other projects.  Indeed, comparing two versions of "the same
routine" often highlights the assumptions or exploits in each.

To sum up, I think it behooves every developer to examine the
libraries that they use so they understand the costs and
vulnerabilities associated with each of those function
invocations that they often make so "casually".  And, if you
tend to use more "esoteric" characteristics of those routines,
you might surprise yourself to discover the cases where they
*fail* (my favorites tend to be things along the lines of
printf("%.*Lf", MAXINT, 1.0L/3.0) -- which, unless
I have misremembered the syntax, here, *should* work on all
conforming implementations!  See how *yours* fares  :> )

Hi Roberto (and Paul),

On 6/15/2011 11:32 AM, Roberto Waltman wrote:
> "Paul E. Bennett" wrote:
>> steve wrote:
>>
>>> What percentage would you say your current project consists of
>>> software reuseable items?
>>>
>>> By software reusable items I mean something that you acquired
>>> (purchased, freeware, shareware, licensed etc) which you had no
>>> responsibility in the development, construction, or testing. You
>>> reused them because the documentation was sufficient to convince you
>>> of its quality.
>>
>> Many of us would have problems recognising that as a definition of re-use.
>> Your definition is more like COTS* usage to me.
>
> Agree.

While I understand (both) your point, I can see the OP's intent,
here.  How do you differentiate a third party COTS product from
a product that the guy who sat in your cubicle *last* year wrote!?

A necessary prerequisite for reuse is a complete "understanding"
(documentation) of what that code does and is intended to do and
what it *won't* do.  And, in which environments those claims
apply.

>> If you really want to talk about re-use then perhaps you can look to a
>> better definition first and then ask the question again.
>>
>> By my notion of re-use I would put the percentage in my projects at
>> somewhere between 30% and 60% depending on the project being undertaken.
>>
>> *Commercial Off-the-shelf
>
> Same for me - In my current project I would say 70% of the code is
> reused.  Most of it is in common libraries developed in-house for
> other projects in the same product line.

I think that is a ^^^^^^^^^^^^^^^^^^^^^^^ key factor.  If your
projects/products are similar, then you can benefit from reusing
code written (designed) to solve a "very similar problem" in
a sister product.  Because the needs *tend* to be the same and the
*environment* tends to be the same.

I wouldn't, for example, feel comfortable porting code from
a data logger that was designed to run on stripped down
hardware to a "PC platform"...

On 15/06/11 22:55, Tim Wescott wrote:
> On 06/15/2011 01:25 PM, David Brown wrote:
>> On 15/06/11 21:39, Tim Wescott wrote:
>>> On 06/15/2011 08:20 AM, David Brown wrote:
>>>> On 15/06/2011 16:03, steve wrote:
>>>>> What percentage would you say your current project consists of
>>>>> software reuseable items?
>>>>>
>>>>> By software reusable items I mean something that you acquired
>>>>> (purchased, freeware, shareware, licensed etc) which you had no
>>>>> responsibility in the development, construction, or testing. You
>>>>> reused them because the documentation was sufficient to convince you
>>>>> of its quality.
>>>>
>>>> 0 percent.
>>>>
>>>> If you use software from a third party, then /you/ are responsible for
>>>> testing and otherwise qualifying it for use in /your/ system.
>>>>
>>>> Getting the software parts from somewhere you consider reliable will
>>>> certainly /reduce/ the level of testing you need to do to be happy with
>>>> its quality, but it does not eliminate it.
>>>>
>>>> If you buy your chips from a reliable supplier, and they come with good
>>>> datasheets qualifying temperature ranges, power requirements, etc.,
>>>> does
>>>> that mean you don't have to test your boards when you use them? It
>>>> might
>>>> mean you don't have to test over a wide temperature or voltage range,
>>>> but you still have to test the boards.
>>>>
>>>> The same thing applies to software components.
>>>
>>> You write your libraries from scratch, or do all your work in assembly?
>>>
>>
>> I don't use much third-party code (for small systems - the balance is
>> different for larger systems). Obviously there is some library code with
>> the compiler (especially language support code), though I don't tend to
>> make much use of the standard C library.
>>
>> But my point is not that I or anyone else doesn't use third-party code -
>> just that if I use third-party code in my project, then /I/ have a
>> responsibility for testing, and making sure it is suitable for the job.
>> The OP seems to think it is appropriate to take other people's code and
>> use it, and let /them/ be responsible for any failures.
>>
>> Obviously the amount of testing and qualification you do with the
>> third-party code depends on its source, and the type and quality of the
>> code. You can generally rely on your compiler's integer division
>> routines to give the correct result - but you would want to be a lot
>> more careful about some code you found lying around on the internet or
>> bought from a vendor you are not sure about (price is no indication of
>> quality).
>
> I was only yanking your chain about your "0 percent" -- the rest of it I
> agree with, wholeheartedly.
>

Perhaps, but it's a fair point.  Many people don't realise that a 
compiler library (both the support library, and the C library) is also 
third-party code.

> Even something as supposedly intrinsic as floating point code can't be
> trusted -- the early Code Composter library for the TMS320F28xx
> processor had a bug where numbers that were very close to an integer
> power of 2 would get doubled (or halved, I can't remember) when rounding
> to an integer, so 0.999999999 would end up as 2.0 (or 0.5), 254.99999999
> would end up as 512 (or 128), etc.
>

/All/ versions of Code Composter (can I borrow that name?) have a 
serious design flaw - uninitialised statically allocated data, which 
should be initialised to 0 according to the C standards, is left 
uninitialised.  That caused me a lot of pain before I discovered it was 
the CCS compiler that was flawed, not my code.

> It was a royal pain in the behind, and fortunately they fixed it. But
> the only comment I can add to your "you can't trust nothin'" is "that's
> right -- and you can't trust anything else, either".
>

On 06/15/2011 05:03 PM, David Brown wrote:
> On 15/06/11 22:55, Tim Wescott wrote:
<snip>
>
> /All/ versions of Code Composter (can I borrow that name?) have a
> serious design flaw - uninitialised statically allocated data, which
> should be initialised to 0 according to the C standards, is left
> uninitialised. That caused me a lot of pain before I discovered it was
> the CCS compiler that was flawed, not my code.
>
>> It was a royal pain in the behind, and fortunately they fixed it. But
>> the only comment I can add to your "you can't trust nothin'" is "that's
>> right -- and you can't trust anything else, either".
>>

I know that the uninitialized statically allocated data is 
work-aroundable, but you have to write some startup code to make it happen.

Unfortunately I'm not the one who wrote the startup code -- I was 
software lead on that, doing head-in-the-clouds control loop and 
communications protocol code, while a really sharp fellow I worked with 
actually found the problem and made it go away.

And yes -- you're welcome to use that name.  I actually tried to learn 
where the thing stored it's splash screen, so I could turn


    C O D E  C O M P O S E R

into

                        T
                        v
    C O D E  C O M P O S E R

Reason (and a tight project schedule) intervened, however.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

Hi David,

On 6/15/2011 5:03 PM, David Brown wrote:

> /All/ versions of Code Composter (can I borrow that name?) have a
> serious design flaw - uninitialised statically allocated data, which
> should be initialised to 0 according to the C standards, is left
> uninitialised. That caused me a lot of pain before I discovered it was
> the CCS compiler that was flawed, not my code.

Huh?  Can't you fix that in crt0.s?

On Jun 15, 11:20=A0am, David Brown <da...@westcontrol.removethisbit.com>
wrote:
> On 15/06/2011 16:03, steve wrote:
>
> > What percentage would you say your current project consists of
> > software reuseable items?
>
> > By software reusable items I mean something that you acquired
> > (purchased, freeware, shareware, licensed etc) which you had no
> > responsibility in the development, construction, or testing. You
> > reused them because the documentation was sufficient to convince you
> > of its quality.
>
> 0 percent.
>
> If you use software from a third party, then /you/ are responsible for
> testing and otherwise qualifying it for use in /your/ system.
>
> Getting the software parts from somewhere you consider reliable will
> certainly /reduce/ the level of testing you need to do to be happy with
> its quality, but it does not eliminate it.
>
> If you buy your chips from a reliable supplier, and they come with good
> datasheets qualifying temperature ranges, power requirements, etc., does
> that mean you don't have to test your boards when you use them? =A0It
> might mean you don't have to test over a wide temperature or voltage
> range, but you still have to test the boards.
>
> The same thing applies to software components.

But to qualify it you have to test the boards, not the chip. There is
a difference between system validation and component testing, when you
buy a chip where you involved in the gate level testing? No. Do you
perform gate level testing of the chip when you get it, no. You treat
the chip as a black box, completely unaware of it's development cycle,
layout history, or testing. You make no attempt to repeat the
suppliers chip level testing and generally don't have any desire to
know what they did (and is generally intellectual property so  you
couldn't find out if you wanted to).

What you do is you plug it in your system and verify your system meets
customer specs. Your job is to verify it performs while interacting
with other components in the system. That is your testing
responsibility.

Same would be true with reuseable code, you test the system, not the
inners of the reuseable code. You qualify it the same way without the
line by line testing. So your argument isn't valid.

On Jun 15, 1:55=A0pm, "Paul E. Bennett" <Paul_E.Benn...@topmail.co.uk>
wrote:
> steve wrote:
> > What percentage would you say your current project consists of
> > software reuseable items?
>
> > By software reusable items I mean something that you acquired
> > (purchased, freeware, shareware, licensed etc) which you had no
> > responsibility in the development, construction, or testing. You
> > reused them because the documentation was sufficient to convince you
> > of its quality.
>
> Many of us would have problems recognising that as a definition of re-use=
.
> Your definition is more like COTS* usage to me.
>
> If you really want to talk about re-use then perhaps you can look to a
> better definition first and then ask the question again.
>

no, I knew exactly what I was asking.

On Jun 15, 3:42=A0pm, Tim Wescott <t...@seemywebsite.com> wrote:
> On 06/15/2011 07:03 AM, steve wrote:
>
> > What percentage would you say your current project consists of
> > software reuseable items?
>
> > By software reusable items I mean something that you acquired
> > (purchased, freeware, shareware, licensed etc) which you had no
> > responsibility in the development, construction, or testing. You
> > reused them because the documentation was sufficient to convince you
> > of its quality.
>
> I'm currently reusing software that I've written. =A0I'd like to say that
> I'm not modifying it, but it is getting tweaked a bit. =A0Beyond that I'm
> coding in C++ and using the libraries that come with the gcc ARM compiler=
.
>
Yes I would consider the libraries true reuse as well as types.

Tweaking your own code, not reuse, maybe junk yard salvage :). Much
better then starting from scratch though.

On Jun 15, 5:41=A0pm, Don Y <nowh...@here.com> wrote:
> Hi Roberto (and Paul),
>
> On 6/15/2011 11:32 AM, Roberto Waltman wrote:
>
>
>
>
>
> > "Paul E. Bennett" wrote:
> >> steve wrote:
>
> >>> What percentage would you say your current project consists of
> >>> software reuseable items?
>
> >>> By software reusable items I mean something that you acquired
> >>> (purchased, freeware, shareware, licensed etc) which you had no
> >>> responsibility in the development, construction, or testing. You
> >>> reused them because the documentation was sufficient to convince you
> >>> of its quality.
>
> >> Many of us would have problems recognising that as a definition of re-=
use.
> >> Your definition is more like COTS* usage to me.
>
> > Agree.
>
> While I understand (both) your point, I can see the OP's intent,
> here. =A0How do you differentiate a third party COTS product from
> a product that the guy who sat in your cubicle *last* year wrote!?
>
> A necessary prerequisite for reuse is a complete "understanding"
> (documentation) of what that code does and is intended to do and
> what it *won't* do. =A0And, in which environments those claims
> apply.

But why? We don't require this strict "complete understanding" for HW
components. Some HW datasheets are wrong, components don't work as
stated, they are returned for failure analysis etc etc. But we deal
with it. We build systems that work because we test at a high level.
We still build reliable systems with all these headaches.

For SW, the attitude (from the responses here and my personal
experience) is it has to be guaranteed to work 100% or otherwise I'll
just code it myself. And this attitude exists for 3rd party SW as well
as for 1 year old software done in house by the guy who sat in the
cube next to you, like you mentioned.

There is a process to deal with defective COTS SW like defective HW.

>
> >> If you really want to talk about re-use then perhaps you can look to a
> >> better definition first and then ask the question again.
>
> >> By my notion of re-use I would put the percentage in my projects at
> >> somewhere between 30% and 60% depending on the project being undertake=
n.
>
> >> *Commercial Off-the-shelf
>
> > Same for me - In my current project I would say 70% of the code is
> > reused. =A0Most of it is in common libraries developed in-house for
> > other projects in the same product line.
>
> I think that is a ^^^^^^^^^^^^^^^^^^^^^^^ key factor. =A0If your
> projects/products are similar, then you can benefit from reusing
> code written (designed) to solve a "very similar problem" in
> a sister product. =A0Because the needs *tend* to be the same and the
> *environment* tends to be the same.
>

Yes exactly, that is the typical "reuse" I normally encounter.

Only occurs if very similar projects, same product line or small
enhancements to a product.  I don't call that reuseable code. I mean,
what else would you do if you have to enhance a product line, throw
everything out and start over? Any code, no matter how badly it's
written can be used in situations like that. Call it what it really
is, a point (custom) solution modified to be another point solution
for a similar product.

Reuseable code is code that can be used in a completely different
product as is.