In message <4931E628.D022C31A@yahoo.com>, CBFalconer 
<cbfalconer@yahoo.com> writes
>Chris H wrote:
>>
>... snip ...
>>
>> No but that does not make it secret. The methods certainly are NOT
>> secret.
>>
>> Many highly qualified and experienced people have in seen the source
>> code of both the main test suites. All the commercial compiler
>> companies for a start.  All of the test and validation houses have.
>>
>> It is just that the source (and IP is not FOSS)
>
>And that gives those firms the right to say "We have applied our
>secret tests to it, and consider it accurate".  No more.

Those tests are NOT secret.
They are just not FOSS

-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In message <4931E595.2D15AD64@yahoo.com>, CBFalconer 
<cbfalconer@yahoo.com> writes
>Walter Banks wrote:
>> CBFalconer wrote:
>>
>... snip ...
>>
>>> And my point is that such 'secret' tests are useless.  You can get
>>> away with this for something like an  editor, where the failures
>>> are fairly evident.  But you can't just say "I have tested this"
>>> without some reasonable backup.
>>
>> No one has. There are two main forms of testing. Both use back
>> up documentation.  Richard is referring to tests that require
>> producing and certifying a document that details a series of
>> measurements resulting from a series of prescribed procedures
>> following a publicly available standards document.
>
>And Chris is referring to some nebulous tests

No. Very clearly defined tests.

>that are not
>publicized,

  Very well known

> cannot be tested and criticized,

Have been tested and criticized

>under threat of
>copyright suits,

All Sw has a license particularly FOSS

> and should be accepted by the docile 'rabble'.

Yes.

-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In message <4931E48F.C78A627B@yahoo.com>, CBFalconer 
<cbfalconer@yahoo.com> writes
>Chris H wrote:
>>
>... snip ...
>>
>> There is a vast amount of qualified and experienced backup  Even
>> some GCC companies use these test suites. The fact that the rabble
>> don't get the source code is neither here not there.
>
>But it is.  Who are you to define 'rabble' anyhow?  If that rabble
>is intended to accept the fact that 'This thing was tested and
>passed' they are entitled to know the details of the testing.  The
>pure existence of a hiding methodology makes the whole process
>highly suspicious.

The method is not AFAIK hidden. Just  the source code.

-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

CBFalconer wrote:

> Walter Banks wrote:
> > CBFalconer wrote:
> >
> ... snip ...
> >
> >> And my point is that such 'secret' tests are useless.  You can get
> >> away with this for something like an  editor, where the failures
> >> are fairly evident.  But you can't just say "I have tested this"
> >> without some reasonable backup.
> >
> > No one has. There are two main forms of testing. Both use back
> > up documentation.  Richard is referring to tests that require
> > producing and certifying a document that details a series of
> > measurements resulting from a series of prescribed procedures
> > following a publicly available standards document.
>
> And Chris is referring to some nebulous tests that are not
> publicized, cannot be tested and criticized, under threat of
> copyright suits, and should be accepted by the docile 'rabble'.

I think you misread the relevant posts.

w..

On Wed, 26 Nov 2008 19:42:24 -0500, CBFalconer <cbfalconer@yahoo.com>
wrote:

>Because, as far as I am concerned, a secret method of validating
>something is totally worthless.  This has the same value as my

The methods of testing an RTOS (which what the OT was all about)
are public (as the standard defines it). But the test program are not
public to you if you are not a customer.

But, and this is important, a customer of a certified software gets
more than you will ever get from some kind of OpenSource(TM) software:
- Test suits
- Paperwork
- design information
- source code

Just take a single simple Linux driver and try to make a full boundary
test.
-- 
42Bastian
Do not email to bastian42@yahoo.com, it's a spam-only account :-)
Use <same-name>@monlynx.de instead !

On Mon, 24 Nov 2008 16:02:31 -0500, CBFalconer <cbfalconer@yahoo.com>
wrote:

>Not Really Me wrote:
>> 
>... snip ...
>> 
>> You should also be aware, that although FreeRTOS has been ported
>> to a number of processors, according to their web site SafeRTOS
>> has only been certified on one.  If you are not using that
>> specific chip/compiler combination your expense is likely to be
>> much higher.
>
>And I have my doubts.  Where is the source of this test suite?  To
>what independant international standard does it adhere?  If this
>stuff is available, why can't the users adapt it to other systems,
>and publish their work?  Etc. etc.

Despite the legal issues: You may not alter the certified version. At
least for Sciopta it is checksummed and a bit changed voids the
certificate.

-- 
42Bastian
Do not email to bastian42@yahoo.com, it's a spam-only account :-)
Use <same-name>@monlynx.de instead !

On Mon, 24 Nov 2008 08:43:36 -0700, "Not Really Me"
<scott@validatedQWERTYsoftware.XYZZY.com> wrote:

>Disclaimer:  We produce the Validation Suites for uCOS, so my opinion may 
>not be completely unbiased.
>
>You should also be aware, that although FreeRTOS has been ported to a number 
>of processors, according to their web site SafeRTOS has only been certified 

There is a small difference between uC/OS on one side and SafeRTOS and
SCIOPTA on the other: Those two are certified. That is, you may take
it as black box plus you can rely on a certified scheduling. Whereas
with the validation suite, I have to take the RTOS as well as my code
to certify it.

>on one.  If you are not using that specific chip/compiler combination your 
>expense is likely to be much higher.

As of IEC61508, the enviromnent must be fixed for a certification
which includes tools such as compiler,assembler and linker.

But a re-certification with different tools (if they are valid for
safety related systems) does not cost as much as the original one.
Mainly because all the paperwork and test-suites are already done.
-- 
42Bastian
Do not email to bastian42@yahoo.com, it's a spam-only account :-)
Use <same-name>@monlynx.de instead !

"FreeRTOS.org" wrote:
> "CBFalconer" <cbfalconer@yahoo.com> wrote:
>> "FreeRTOS.org" wrote:
>>
>> ... snip ...
>>
>>> Just to be clear - this thread was about validating components
>>> such as an RTOS for safety work.  *Nothing* I have written in
>>> this thread is related to language compliance testing for
>>> compilers.
>>
>> It still is.  Compilers are just a typical example of a component.
> 
> [trying out my new Motzarella account - thanks for the tip]
> 
> I agree compilers are a component, but I'm still talking about
> something different.  I'm talking about testing components to
> international safety standards, as published by, ratified by,
> inspected by and audited by 'official' bodies.  Chris is talking
> about language compliance testing, which is not 'governed' in the
> same way.   In any case a compiler can be 100% compliant with a
> standard (if you can tie the standard down 100%), and still
> produce bum code, (as discussed in the ARM IDE thread) so being
> language compliant makes little difference to how you would
> validate the use of a compiler in a particular application - but
> PLEASE lets no go there again - not in this thread anyway.

Well, I think everyone has expressed their opinion, at least once,
so I see no need to continue it.  :-)

-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.

"CBFalconer" <cbfalconer@yahoo.com> wrote in message 
news:4931E682.EFA291@yahoo.com...
> "FreeRTOS.org" wrote:
>>
> ... snip ...
>>
>> Just to be clear - this thread was about validating components
>> such as an RTOS for safety work.  *Nothing* I have written in
>> this thread is related to language compliance testing for
>> compilers.
>
> It still is.  Compilers are just a typical example of a component.

[trying out my new Motzarella account - thanks for the tip]

I agree compilers are a component, but I'm still talking about something 
different.  I'm talking about testing components to international safety 
standards, as published by, ratified by, inspected by and audited by 
'official' bodies.  Chris is talking about language compliance testing, 
which is not 'governed' in the same way.   In any case a compiler can be 
100% compliant with a standard (if you can tie the standard down 100%), and 
still produce bum code, (as discussed in the ARM IDE thread) so being 
language compliant makes little difference to how you would validate the use 
of a compiler in a particular application - but PLEASE lets no go there 
again - not in this thread anyway.

-- 
Regards,
Richard.

+ http://www.FreeRTOS.org & http://www.FreeRTOS.org/shop
17 official architecture ports, more than 6000 downloads per month.

+ http://www.SafeRTOS.com
Certified by T&#4294967295;V as meeting the requirements for safety related systems.

"FreeRTOS.org" wrote:
> 
... snip ...
> 
> Just to be clear - this thread was about validating components
> such as an RTOS for safety work.  *Nothing* I have written in
> this thread is related to language compliance testing for
> compilers.

It still is.  Compilers are just a typical example of a component.

-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.