FreeRTOS / SafeRTOS in a Medical Device

Thanks to everyone for your responses in this thread.  If no one
minds, I will combine my responses to all, in this post.

First of all, a bit of history... I have done a lot of medical device
development work but not in the last five years or so.  Hence I may
have lost track a bit, regarding the current thinking of the FDA on
this topic, although I am well familiar with the FDA guidances
including the one on OTS software, and they don't seem to have changed
markedly in the last several years.

Back in the mid-late 1990's, I worked on a Class 2 medical device that
was based (I swear I'm not making this up) on MS-DOS 5.0.  The company
had no trouble getting that past FDA.  They had trouble getting other
aspects of the development past the FDA, but not the operating system.
That device, as well as the device I'm currently working on, was
"substantially equivalent" to other devices on the market, and so the
companies involved did seek and are seeking approved via the 510k
route and not the PMA route.

Other medical device projects I have worked on used VxWorks, LynxOS,
and home-grown solutions.  None were "certified", per se, by FDA.
However we were able to "validate for intended use" by testing and by
showing that they were mature products in the industry, and that their
manufacturers had effective support and defect-repair systems in
place.  We were never asked for the full documentation and design
controls package, including design history file, for any of the
commercial operating systems, and of course we would never have been
able to provide it had we been asked.

Perhaps all of this ties in with what one means by "safety critical".
This current project is to develop a Class 2 device, which will be
presented for approval via the 510k route, and we are going to argue
that the software is of a Moderate Level of Concern (we may not get
away with that one and I'm certainly going to encourage the team lead
to run that one past the FDA sooner rather than later).  So perhaps it
doesn't rise to the definition of "safety-critical" that would trigger
the need for all the design controls documentation of the OS (and
would probably trigger the need for a PMA as well).

Perhaps in the last five years or so, the FDA has solidified its
expectations regarding OTS software, and we certainly expect to work
with them to clarify those expectations and make sure we meet them.
But, as I said, in the past none of the products I have worked on,
using home-grown, VxWorks, or LynxOS, have had any particular trouble
getting the OS's past FDA review, even if the OS's were not
"certified".  Anyway, back then, there really wasn't any such thing as
a "certified" VxWorks... is there such a beast today?  And, the
version of LynxOS we used was the plain vanilla one, not the
LynxOS-178 or even the -SE.

(Neither VxWorks nor LynxOS will run on the planned platform, so all
of the VxWorks / LynxOS discussion is moot, but I just wanted to point
out that in past projects, those OS's weren't "certified" per se, and
we didn't have any particular problems due to that.)

Now, on to some of the points that you all have generously raised...

>> First question:- Do you really need an OS for the project?

As a practical matter, yes.  We have done some architecture design and
we expect the software to run in at least a half dozen separate tasks
at different priorities.  It's always possible to roll one's own, but
as we all know, it makes no sense when commercial products are
available at low cost.  For example, OpenRTOS at I think something
like $4000 or $5000 a license would be a no-brainer for this product.
SafeRTOS at $60,000, amortized over the quantities this company is
talking about, would require some serious thought.

>> Second question:- Do you have a really sound development
>> process that gives you the evidence for what was done at
>> each stage of the development?

That's a separate issue that is being addressed.  Suffice to say that
it's an area that is a work in progress.  However, it needs to be
there for all of the design aspects of the product, not just the OS.

>> You need to ensure you create the right documentation at
>> each stage and in the right order. You need to do risk
>> assessments for the system and prove the dependability
>> of the resultant design through adequate review and testing.
>> Make sure your process gets the bugs out early (before you
>> start in on the detail design preferably).

We are in the process now of establishing the policies and procedures
that control this.  Certainly everything you suggest is part of the
plan.  But, all of this needs to apply to all of the software, not
just the small percentage that is the OS.  See below...

>> I think the single project license is actually $45k

This product is intended to grow into a family of related products, so
we'll likely need the unlimited-projects license.

Several of you have pointed out, rightfully so, that the kind of
validation required for the operating system of a truly
"safety-critical" product (depending on how one defines the term) is
likely to cost more than $60,000.  That's certainly true enough.
However, ALL of the software for this system is going to have to
undergo an appropriate level of validation, and the operating system
is a small percentage of the overall code base.  Therefore, I claim
that we could apply the same validation procedures to FreeRTOS (since
we have the source code) as to the rest of the software in the
product, and the only difference is that we wouldn't have the full
development life cycle documentation for the OS.  That wouldn't have
been any particular problem "back in the day" for this class of
medical device.  Maybe FDA has gotten more strict about that stuff in
the last few years.

You all have pointed out the advantages of the SafeRTOS Design
Assurance Package, mainly that it provides all of the validation
evidence, suitable for truly safety-critical systems, for a one-time
payment.  My question is, do we need it at that level?  This isn't a
DO-178B Level A device like a fly-by-wire system (but, does SafeRTOS's
"certification" really rise to that level of demonstrable reliability
in the eyes of the FAA, which is considerably stricter about such
things than the FDA?).

In conclusion, I would like to apologize for characterizing the
SafeRTOS cost as "wildly expensive".  I know that sounds like I was
criticizing, and that wasn't the intent.  I can see that, for a truly
safety-critical system, it's a good deal.  I'm just not at all
convinced (yet) that we'll need it at that level.

"C. J. Clegg" <answer.in.newsgroup@no.spam> wrote in message 
news:chegi491g2a18d3b4qu832r7arqhvlue40@4ax.com...
>
> Thanks to everyone for your responses in this thread.  If no one
> minds, I will combine my responses to all, in this post.

Hi C. J. - could you send me a message to r (_dot_) barry {at} 
freertos.org - I would be interested in having a FreeRTOS.org related chat. 
I know this is not the done thing in these groups and am not suggesting that 
we don't continue this interesting thread here too.  I also pledge not to 
pass your email onto any of the commercial guys unless I have your 
permission to first :o)

-- 
Regards,
Richard.

+ http://www.FreeRTOS.org & http://www.FreeRTOS.org/shop
17 official architecture ports, more than 6000 downloads per month.

+ http://www.SafeRTOS.com
Certified by T&#4294967295;V as meeting the requirements for safety related systems. 

On Nov 22, 5:28=A0pm, "FreeRTOS.org" <noem...@given.com> wrote:
> "C. J. Clegg" <answer.in.newsgr...@no.spam> wrote in messagenews:chegi491=
g2a18d3b4qu832r7arqhvlue40@4ax.com...
>
>
>
> > Thanks to everyone for your responses in this thread. =A0If no one
> > minds, I will combine my responses to all, in this post.
>


Interesting discussion - worth noting I also downloaded the SafeRTOS
pricing and it is only $45k for a project license........

On Sat, 22 Nov 2008 17:28:47 GMT, "FreeRTOS.org" <noemail@given.com>
wrote:

>could you send me a message to r (_dot_) barry {at} 
>freertos.org - I would be interested in having a FreeRTOS.org related chat. 

I could do that... but there really isn't anything I can contribute
privately that I can't contribute publicly, and there's nothing I can
say that I wouldn't want the commercial guys to see.

I'm all for giving the widest possible audience the benefit of these
discussions, to the extent that they are interested...

>>could you send me a message to r (_dot_) barry {at}
>>freertos.org - I would be interested in having a FreeRTOS.org related 
>>chat.
>
> I could do that... but there really isn't anything I can contribute
> privately that I can't contribute publicly,

...but there are some things I can contribute privately that I cannot on a 
public forum.

-- 
Regards,
Richard.

+ http://www.FreeRTOS.org & http://www.FreeRTOS.org/shop
17 official architecture ports, more than 6000 downloads per month.

+ http://www.SafeRTOS.com
Certified by T�V as meeting the requirements for safety related systems. 

In message <gg96e1$uqf$1@aioe.org>, Bresco <bresco@mixmaster.org> writes
>
>"Chris H" <chris@phaedsys.org> wrote in message
>news:azcwaiANTBKJFAOd@phaedsys.demon.co.uk...
>> I think you may find that it will be more expensive to try and do a
>> certified Linux system.
>>
>
>I'm not advocating neither Linux w/ RT patch nor eCos, just summing up the
>possibilities. For non-medical applications these two will suffice in 99% of
>all cases.

You mean non-safey-critical

there are many systems that require validation other than medical 
systems.
-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In message <HFYVk.91394$E41.41118@text.news.virginmedia.com>, 
FreeRTOS.org <noemail@given.com> writes
>>>could you send me a message to r (_dot_) barry {at}
>>>freertos.org - I would be interested in having a FreeRTOS.org related
>>>chat.
>>
>> I could do that... but there really isn't anything I can contribute
>> privately that I can't contribute publicly,
>
>...but there are some things I can contribute privately that I cannot on a
>public forum.
>
Of course if this was FOSS..... :-)

(I'll get me coat :-)
-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

FreeRTOS.org wrote:

> Done and included in the DAP.
> 
>> Make sure your process gets the
>> bugs out early (before you start in on the detail design preferably).
> 
> Getting all the bugs out before starting the detailed design, now that
> would
> be nice.  I'm assuming your talking about the system engineering bugs
> ;o)

Yes it would be very nice if everyone else would do just that. The
majority of the bugs in a system happen in the requirements
specification stage. Getting all of those out before starting detailed
design will always improve the technical detailed design. Getting
correct designs has always been and will always be by having a good
process. There is no magic language, no magic OS that does it all for
you and you have to expend some verification effort even if you are
buying in a SafetyRTOS type product.

-- 
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-811095
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

In message <6osqgnF57re2U1@mid.individual.net>, Paul E. Bennett 
<Paul_E.Bennett@topmail.co.uk> writes
>FreeRTOS.org wrote:
>
>> Done and included in the DAP.
>>
>>> Make sure your process gets the
>>> bugs out early (before you start in on the detail design preferably).
>>
>> Getting all the bugs out before starting the detailed design, now that
>> would
>> be nice.  I'm assuming your talking about the system engineering bugs
>> ;o)
>
>Yes it would be very nice if everyone else would do just that. The
>majority of the bugs in a system happen in the requirements
>specification stage.

There is a lot of data to back this up. Incidentally as Paul knows on 
Safety critical systems about 50% of the problems come from incorrect 
specifications. Less than 14% from the implementation phase regardless 
of language used.

In a good, well defined, process the choice of language has minimal 
impact.

>Getting all of those out before starting detailed
>design will always improve the technical detailed design. Getting
>correct designs has always been and will always be by having a good
>process.

So very true. I have seen data to conform this. However many will always 
try and subvert this. In fact some of the systems currently in vogue run 
counter to this. They will, eventually, learn the hard way

> There is no magic language, no magic OS that does it all for
>you and you have to expend some verification effort even if you are
>buying in a SafetyRTOS type product.

If you buy in a validated RTOS and use validated compilers you STILL 
have to have a good process for the rest of the project. You need to 
put all the rest of the code though a rigorous process.  It is just that 
the RTOS will have been done for you. Saving much time

Using certified compilers likewise saves you a hell of a lot of time and 
effort.  The cost of the compiler fades in to insignificance compered to 
the other costs.





-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

C. J. Clegg wrote:

[%X]

> Now, on to some of the points that you all have generously raised...
> 
>>> First question:- Do you really need an OS for the project?
> 
> As a practical matter, yes.  We have done some architecture design and
> we expect the software to run in at least a half dozen separate tasks
> at different priorities.  It's always possible to roll one's own, but
> as we all know, it makes no sense when commercial products are
> available at low cost.  For example, OpenRTOS at I think something
> like $4000 or $5000 a license would be a no-brainer for this product.
> SafeRTOS at $60,000, amortized over the quantities this company is
> talking about, would require some serious thought.

So long as that makes sense to your product then fine. There have been
times when I have seen people have terrific problems with an OS on a
project when it would have been easier without. My own foray into
Medical Systems was on Anaesthesia Ventilators.
 
>>> Second question:- Do you have a really sound development
>>> process that gives you the evidence for what was done at
>>> each stage of the development?
> 
> That's a separate issue that is being addressed.  Suffice to say that
> it's an area that is a work in progress.  However, it needs to be
> there for all of the design aspects of the product, not just the OS.

When you get your process together don't forget to test it out and make
sure it catches problems before they get too far. Do regular audits of
the process and ensure everyone in the team is following it.

>>> You need to ensure you create the right documentation at
>>> each stage and in the right order. You need to do risk
>>> assessments for the system and prove the dependability
>>> of the resultant design through adequate review and testing.
>>> Make sure your process gets the bugs out early (before you
>>> start in on the detail design preferably).
> 
> We are in the process now of establishing the policies and procedures
> that control this.  Certainly everything you suggest is part of the
> plan.  But, all of this needs to apply to all of the software, not
> just the small percentage that is the OS.  See below...

Definitely.
 
>>> I think the single project license is actually $45k
> 
> This product is intended to grow into a family of related products, so
> we'll likely need the unlimited-projects license.
> 
> Several of you have pointed out, rightfully so, that the kind of
> validation required for the operating system of a truly
> "safety-critical" product (depending on how one defines the term) is
> likely to cost more than $60,000.  That's certainly true enough.
> However, ALL of the software for this system is going to have to
> undergo an appropriate level of validation, and the operating system
> is a small percentage of the overall code base.  Therefore, I claim
> that we could apply the same validation procedures to FreeRTOS (since
> we have the source code) as to the rest of the software in the
> product, and the only difference is that we wouldn't have the full
> development life cycle documentation for the OS.  That wouldn't have
> been any particular problem "back in the day" for this class of
> medical device.  Maybe FDA has gotten more strict about that stuff in
> the last few years.

If the RTOS is really such a small part then it does not seem like it
will be much more burden on your efforts with the rest of the system.
Hope you have a team for the OS reverse engineering effort as this may
be the only way to be certain it really is sound. I presume that
FreeRTOS might be in something like C (don't know for sure as never used
it - but is a reasonable guess I think) in which case getting the code
through a LINT or similar tool configured for MISRA-C rules might help
you out a bit. Don't forget in all this effort that the "MK1 eyeball" is
also a very good tool (ie: spending some time just looking at the code -
asking oneself "is it elegant?" "does it logically scan?" "does the
coded function meet with the requirements?" and a host of other related
questions).

> You all have pointed out the advantages of the SafeRTOS Design
> Assurance Package, mainly that it provides all of the validation
> evidence, suitable for truly safety-critical systems, for a one-time
> payment.  My question is, do we need it at that level?  This isn't a
> DO-178B Level A device like a fly-by-wire system (but, does SafeRTOS's
> "certification" really rise to that level of demonstrable reliability
> in the eyes of the FAA, which is considerably stricter about such
> things than the FDA?).

I have seen projects come unstuck when the developers assumed the product
would not be safety critical but in use it turns out should have been a
SIL1 system. At least you have some appreciation of the kind of Hazards
and Risks involved (from experience). This is why all system developers
should start with a full list of the Hazards that might be faced by
their system implementation and design out the risk level accordingly
(fully documenting as you go). I tend to begin by assuming I will
develop for SIL4 until proven otherwise.

[%X]

-- 
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-811095
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************