Group Design

Wow!  Designing in a group is strange.  I'm helping with the design of a ventilator and it is a unique experience.  We have people of many skill levels and talents, many I don't even know.  The design seems to ping pong around based on the hot button issue of the day.  They already have someone to do board level design, but he has a day job, so the work gets done a bit helter skelter.  I mostly try to identify potential bugs.  

Most of the people are in the UK with a couple in the EU.  I'm in the US along with one other person.  

We have semiweekly meetings on zoom or other sites.  There is no strong organization.  

I'm surprised as much is getting done as it has been.  The original design used an Arduino and once the lack of I/Os became a hindrance we are switching to an ARM, but nothing remotely like a proper spec was generated and the processor initially picked is now out of pins.  The first rev of the main board has many issues including the isolation of the heat sink tabs by thermal relief connections.  

I literally have no idea how they are ever going to get this thing past any sort of approval process.  No one working on the project even seems to understand what is required.  

I guess I'm a bit frustrated by the lack of organization.  I'd like to step away, but the person acting as a leader seems to want me in the group.  

There is some work of my own I should do, so maybe I'll stick around to get to rev 2 of the board and then step out.  

I guess I'm used to working on my own. 

-- 

  Rick C.

  - Get 1,000 miles of free Supercharging
  - Tesla referral code - https://ts.la/richard11209

Rick C <gnuarm.deletethisbit@gmail.com> writes:
> Wow!  Designing in a group is strange.  I'm helping with the design of
> a ventilator and it is a unique experience.

Sounds like fun, but why a ventilator?  Is the ventilator shortage still
happening and are there more urgent shortages?  I thought there were
already a bunch of ventilator projects.

What is the software going to be like, and is it being taken care of?
This sounds like a good opportunity to write some Ada code.

On Tuesday, July 7, 2020 at 10:02:45 PM UTC-4, Paul Rubin wrote:
> Rick C <gnuarm.deletethisbit@gmail.com> writes:
> > Wow!  Designing in a group is strange.  I'm helping with the design of
> > a ventilator and it is a unique experience.
> 
> Sounds like fun, but why a ventilator?  Is the ventilator shortage still
> happening and are there more urgent shortages?  I thought there were
> already a bunch of ventilator projects.
> 
> What is the software going to be like, and is it being taken care of?
> This sounds like a good opportunity to write some Ada code.

Yeah, that was one of my first observations that there are tons of ventilator projects.  They nearly all are based on squeezing an Ambu-bag to push air into the patient.  The details of not harming the patient are tricky.  Ventilating a patient while conscious is harder and very few are doing that.  They seem to be having problems with the plumbing at the moment, getting all the right valves in the right places to do the right things.  At the moment there is a lot of hardware right at the patient's mouth.  I'm working to understand what is needed and what isn't.  

There's a valve called a Ruben valve which allows the patient to be connected to the inspiratory limb (branch from the ventilator) or to the expiratory limb (the path for exhaling) without a leak to the outside via a straight through path.  It's not entirely clear just how the details of this valve work.  I have concerns that the valve can be stuck in the inhale position by the pressure in the line.  It doesn't seem to have a way to leak out once pressurized.  

I'm not sure the software is too hard at the higher levels, but there are interface procedures that have to convert differential pressure to flow rate and monitor other marginal sensors.  The O2 sensor is only ~18% accurate.  The guy running the project doesn't seem to understand the difference between accuracy, resolution and noise when we were talking about increasing the resolution of a reading by averaging many samples.  Not that this was a big deal.  The real issue is I'm not sure improving the resolution of the measurement will give him what he wants.  The airflow measurement is very sensitive at the low end.  I'm wondering if there is an issue with the flow being more or less laminar at the low end so that it is not going to be an accurate reading.  The interesting part is they  verified the home brew flow rate sensor by using a commercial flow rate sensor that uses the same technique.  

Whatever.  I'm trying to help without being a drag.  I think sometimes my questions are not what they want to hear.  Maybe I'm too pessimistic.  

The software is C I believe.  I don't think anyone in this crowd is going to use Ada. 

-- 

  Rick C.

  + Get 1,000 miles of free Supercharging
  + Tesla referral code - https://ts.la/richard11209

On 08/07/2020 05:36, Rick C wrote:
> On Tuesday, July 7, 2020 at 10:02:45 PM UTC-4, Paul Rubin wrote:
>> Rick C <gnuarm.deletethisbit@gmail.com> writes:
>>> Wow!  Designing in a group is strange.  I'm helping with the
>>> design of a ventilator and it is a unique experience.
>> 
>> Sounds like fun, but why a ventilator?  Is the ventilator shortage
>> still happening and are there more urgent shortages?  I thought
>> there were already a bunch of ventilator projects.
>> 
>> What is the software going to be like, and is it being taken care
>> of? This sounds like a good opportunity to write some Ada code.
> 
> Yeah, that was one of my first observations that there are tons of
> ventilator projects.  They nearly all are based on squeezing an
> Ambu-bag to push air into the patient.  The details of not harming
> the patient are tricky.  Ventilating a patient while conscious is
> harder and very few are doing that.  They seem to be having problems
> with the plumbing at the moment, getting all the right valves in the
> right places to do the right things.  At the moment there is a lot of
> hardware right at the patient's mouth.  I'm working to understand
> what is needed and what isn't.
> 
> There's a valve called a Ruben valve which allows the patient to be
> connected to the inspiratory limb (branch from the ventilator) or to
> the expiratory limb (the path for exhaling) without a leak to the
> outside via a straight through path.  It's not entirely clear just
> how the details of this valve work.  I have concerns that the valve
> can be stuck in the inhale position by the pressure in the line.  It
> doesn't seem to have a way to leak out once pressurized.
> 
> I'm not sure the software is too hard at the higher levels, but there
> are interface procedures that have to convert differential pressure
> to flow rate and monitor other marginal sensors.  The O2 sensor is
> only ~18% accurate.  The guy running the project doesn't seem to
> understand the difference between accuracy, resolution and noise when
> we were talking about increasing the resolution of a reading by
> averaging many samples.  Not that this was a big deal.  The real
> issue is I'm not sure improving the resolution of the measurement
> will give him what he wants.  The airflow measurement is very
> sensitive at the low end.  I'm wondering if there is an issue with
> the flow being more or less laminar at the low end so that it is not
> going to be an accurate reading.  The interesting part is they
> verified the home brew flow rate sensor by using a commercial flow
> rate sensor that uses the same technique.
> 
> Whatever.  I'm trying to help without being a drag.  I think
> sometimes my questions are not what they want to hear.  Maybe I'm too
> pessimistic.
> 
> The software is C I believe.  I don't think anyone in this crowd is
> going to use Ada.
> 

It sounds like this group is opportunist (I don't mean that in a bad
way) - they see a potential market for a device, with a short time
frame, and they are trying to get something working as fast as possible
before the window closes.  So they grab the first off-the-shelf "rapid"
development kit they can find - an Arduino - and start banging out code
before they've even figured out what they want to do.

For some products, that's fine.  For something medical related, it is
not.  They appear (from your description) to be missing a fundamental
understanding of the task, and haven't got in experts to cover that
base.  And without paperwork covering everything from research, design
calculations, specifications (hardware and software), reviews, project
management, issue tracking, personnel qualifications, development,
testing, certification, production qualification, and a dozen other
topics, there is no way they can get any kind of approval.

I would guess that the group is used to small systems for small
customers who are happy with a demonstration that shows the device
works.  That's great for many things, and gives people solutions that
work when a serious high-level development process would be orders of
magnitude too expensive.

But making things that are relevant to life and death is a different
world entirely.  Some people think that the current Covid situation is
so desperate that /anything/ is better than nothing, so throwing
together a cheap and simple ventilator is helpful - that is very far
from true.

So I hope you get paid up-front, and keep your distance from the group -
be an external consultant, but don't take responsibility for anything
other than your reviews.

(And the choice of language for the software is the least of their
concerns!)

On 2020-07-08 Rick C wrote in comp.arch.embedded:
> Wow!  Designing in a group is strange.  I'm helping with the design of a ventilator and it is a unique experience.  We have people of many skill levels and talents, many I don't even know.

That is the good part of designing with a group. You can have many more talents
and skills than can be combined in a single person. For a medical product like
this you need medical, software, hardware, mechanical and lots more skills. Not
something that can be found in one person very often.

<snip, some project frustrations etc.>

> I literally have no idea how they are ever going to get this thing past any sort of approval process.  No one working on the project even seems to understand what is required.  
>

This is very worrying. As David already explained, documentation and approval
is a very large part of medical device development. There is this famous
Boeing quote about the weight of the paperwork required for a plane. I think
you can extend this to medical devices as well.

Did they at least have a look at some standards that apply to ventilators to
check what will be required? I would start off with IEC 60601-1, IEC 62304
and ISO 80601-2-12, make sure you use the latest versions. But there will be
more.

And this should have been done before the first schematic was drawn or line
of code was written. Some standards put requirements on the development
process and you can not comply with those if you start your certification
efforts after the development is completed.


> I guess I'm a bit frustrated by the lack of organization.  I'd like to step away, but the person acting as a leader seems to want me in the group.  
>
> There is some work of my own I should do, so maybe I'll stick around to get to rev 2 of the board and then step out.  
>

If there has indeed been done nothing on the certification side, my feeling
would be to step away as well.


> I guess I'm used to working on my own. 
>

Yes, than all is in your own hands. ;-)

But the advantage of a team is that you can concentrate on what you are good
at and let others do what they are good at. And if the team is diverse and
skilled enough (and properly led) the end result will be better that that of
a single developer.

And I think a lot of medical device stuff can not be done by a single
person. There are many cases where you need an (expert) reviewer in the
process. Reviewing your own work is less than optimal. ;-)


-- 
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail)

It seems that more and more mathematicians are using a new, high level
language named "research student".

On Wednesday, July 8, 2020 at 3:31:55 AM UTC-4, David Brown wrote:
> On 08/07/2020 05:36, Rick C wrote:
> > On Tuesday, July 7, 2020 at 10:02:45 PM UTC-4, Paul Rubin wrote:
> >> Rick C <gnuarm.deletethisbit@gmail.com> writes:
> >>> Wow!  Designing in a group is strange.  I'm helping with the
> >>> design of a ventilator and it is a unique experience.
> >> 
> >> Sounds like fun, but why a ventilator?  Is the ventilator shortage
> >> still happening and are there more urgent shortages?  I thought
> >> there were already a bunch of ventilator projects.
> >> 
> >> What is the software going to be like, and is it being taken care
> >> of? This sounds like a good opportunity to write some Ada code.
> > 
> > Yeah, that was one of my first observations that there are tons of
> > ventilator projects.  They nearly all are based on squeezing an
> > Ambu-bag to push air into the patient.  The details of not harming
> > the patient are tricky.  Ventilating a patient while conscious is
> > harder and very few are doing that.  They seem to be having problems
> > with the plumbing at the moment, getting all the right valves in the
> > right places to do the right things.  At the moment there is a lot of
> > hardware right at the patient's mouth.  I'm working to understand
> > what is needed and what isn't.
> > 
> > There's a valve called a Ruben valve which allows the patient to be
> > connected to the inspiratory limb (branch from the ventilator) or to
> > the expiratory limb (the path for exhaling) without a leak to the
> > outside via a straight through path.  It's not entirely clear just
> > how the details of this valve work.  I have concerns that the valve
> > can be stuck in the inhale position by the pressure in the line.  It
> > doesn't seem to have a way to leak out once pressurized.
> > 
> > I'm not sure the software is too hard at the higher levels, but there
> > are interface procedures that have to convert differential pressure
> > to flow rate and monitor other marginal sensors.  The O2 sensor is
> > only ~18% accurate.  The guy running the project doesn't seem to
> > understand the difference between accuracy, resolution and noise when
> > we were talking about increasing the resolution of a reading by
> > averaging many samples.  Not that this was a big deal.  The real
> > issue is I'm not sure improving the resolution of the measurement
> > will give him what he wants.  The airflow measurement is very
> > sensitive at the low end.  I'm wondering if there is an issue with
> > the flow being more or less laminar at the low end so that it is not
> > going to be an accurate reading.  The interesting part is they
> > verified the home brew flow rate sensor by using a commercial flow
> > rate sensor that uses the same technique.
> > 
> > Whatever.  I'm trying to help without being a drag.  I think
> > sometimes my questions are not what they want to hear.  Maybe I'm too
> > pessimistic.
> > 
> > The software is C I believe.  I don't think anyone in this crowd is
> > going to use Ada.
> > 
> 
> It sounds like this group is opportunist (I don't mean that in a bad
> way) - they see a potential market for a device, with a short time
> frame, and they are trying to get something working as fast as possible
> before the window closes.  So they grab the first off-the-shelf "rapid"
> development kit they can find - an Arduino - and start banging out code
> before they've even figured out what they want to do.
> 
> For some products, that's fine.  For something medical related, it is
> not.  They appear (from your description) to be missing a fundamental
> understanding of the task, and haven't got in experts to cover that
> base.  And without paperwork covering everything from research, design
> calculations, specifications (hardware and software), reviews, project
> management, issue tracking, personnel qualifications, development,
> testing, certification, production qualification, and a dozen other
> topics, there is no way they can get any kind of approval.
> 
> I would guess that the group is used to small systems for small
> customers who are happy with a demonstration that shows the device
> works.  That's great for many things, and gives people solutions that
> work when a serious high-level development process would be orders of
> magnitude too expensive.
> 
> But making things that are relevant to life and death is a different
> world entirely.  Some people think that the current Covid situation is
> so desperate that /anything/ is better than nothing, so throwing
> together a cheap and simple ventilator is helpful - that is very far
> from true.
> 
> So I hope you get paid up-front, and keep your distance from the group -
> be an external consultant, but don't take responsibility for anything
> other than your reviews.
> 
> (And the choice of language for the software is the least of their
> concerns!)

Pay???  This is open source, volunteer labor.  They have a few companies donating fabrication services.  A sheet metal shop fabricated a few chassis.  I think they may even have a Chinese assembly house donating some board assembly work.  

I started on a different project which was doing a very similar thing and I questioned the need for yet another project.  In the end the guy leading that effort bailed and the group decided to merge with this project that was further along.  

I think I feel awkward because I don't have a defined roll.  It is hard for me to tell if/when my comments are welcome and when they are kibitzing.  

-- 

  Rick C.

  -- Get 1,000 miles of free Supercharging
  -- Tesla referral code - https://ts.la/richard11209

On Wednesday, July 8, 2020 at 8:14:05 AM UTC-4, Stef wrote:
> On 2020-07-08 Rick C wrote in comp.arch.embedded:
> > Wow!  Designing in a group is strange.  I'm helping with the design of a ventilator and it is a unique experience.  We have people of many skill levels and talents, many I don't even know.
> 
> That is the good part of designing with a group. You can have many more talents
> and skills than can be combined in a single person. For a medical product like
> this you need medical, software, hardware, mechanical and lots more skills. Not
> something that can be found in one person very often.
> 
> <snip, some project frustrations etc.>
> 
> > I literally have no idea how they are ever going to get this thing past any sort of approval process.  No one working on the project even seems to understand what is required.  
> >
> 
> This is very worrying. As David already explained, documentation and approval
> is a very large part of medical device development. There is this famous
> Boeing quote about the weight of the paperwork required for a plane. I think
> you can extend this to medical devices as well.
> 
> Did they at least have a look at some standards that apply to ventilators to
> check what will be required? I would start off with IEC 60601-1, IEC 62304
> and ISO 80601-2-12, make sure you use the latest versions. But there will be
> more.
> 
> And this should have been done before the first schematic was drawn or line
> of code was written. Some standards put requirements on the development
> process and you can not comply with those if you start your certification
> efforts after the development is completed.
> 
> 
> > I guess I'm a bit frustrated by the lack of organization.  I'd like to step away, but the person acting as a leader seems to want me in the group.  
> >
> > There is some work of my own I should do, so maybe I'll stick around to get to rev 2 of the board and then step out.  
> >
> 
> If there has indeed been done nothing on the certification side, my feeling
> would be to step away as well.
> 
> 
> > I guess I'm used to working on my own. 
> >
> 
> Yes, than all is in your own hands. ;-)
> 
> But the advantage of a team is that you can concentrate on what you are good
> at and let others do what they are good at. And if the team is diverse and
> skilled enough (and properly led) the end result will be better that that of
> a single developer.
> 
> And I think a lot of medical device stuff can not be done by a single
> person. There are many cases where you need an (expert) reviewer in the
> process. Reviewing your own work is less than optimal. ;-)

It seems the UK has an abbreviated set of requirements for response to COVID-19 and that is what they are mostly working with.  I forget the name of it.  

They do get advice from time to time from professionals in the field.  My main concern is a direction they are currently going in where more and more valves, pressure relief, etc. is being added to the tubing right at the patient.  I'm not sure how it all will be held in place.  That and how they plan to get this through the approval process.  

I had a friend who worked on medical devices for a while before getting out.  He said they required that you "prove" your design will not hurt the patient in any way.  Hard to do with a ventilator.  

I don't mind working on the team.  It's seeing them go in odd directions that bugs me.  This thing will probably cost hundreds of dollars in the final BOM without anything like profit or overhead added in.  People were arguing against adding $0.10 for more pins on a processor.  Then later they decided to not only go with the larger pin count, but bump from an 8 bit AVR to an ARM CM4F adding several dollars.  That sort of inconsistency drives me nuts.  "Every dollar counts!"  Yes it does, so you need to justify the cost *rationally*.  

-- 

  Rick C.

  -+ Get 1,000 miles of free Supercharging
  -+ Tesla referral code - https://ts.la/richard11209

On 2020-07-08 Rick C wrote in comp.arch.embedded:
> On Wednesday, July 8, 2020 at 8:14:05 AM UTC-4, Stef wrote:
>> On 2020-07-08 Rick C wrote in comp.arch.embedded:
>> > Wow!  Designing in a group is strange.  I'm helping with the design of a ventilator and it is a unique experience.  We have people of many skill levels and talents, many I don't even know.
>> 
[...]

[Is there a way you could limit your line length to a more usenet friendly
number?]

> It seems the UK has an abbreviated set of requirements for response to COVID-19 and that is what they are mostly working with.  I forget the name of it.  

How long will this last? If you look at the new cases curves, the UK is well
over the initial peak. https://www.nytimes.com/ has nice graphs of world
cases on the front page.

>
> They do get advice from time to time from professionals in the field.  My main concern is a direction they are currently going in where more and more valves, pressure relief, etc. is being added to the tubing right at the patient.  I'm not sure how it all will be held in place.  That and how they plan to get this through the approval process.  
>

Sounds like they are moving to more and more features instead of sticking to a
simple emergency use device. Feature creep is not uncommon when getting advise
from several medical professionals. ;-)


> I had a friend who worked on medical devices for a while before getting out.  He said they required that you "prove" your design will not hurt the patient in any way.  Hard to do with a ventilator.  
>

That is not entirely true, it is more a risk/benefit analysis. Otherwise you
could not even get a scalpel approved, it will always hurt the patient.

> I don't mind working on the team.  It's seeing them go in odd directions that bugs me.  This thing will probably cost hundreds of dollars in the final BOM without anything like profit or overhead added in.  People were arguing against adding $0.10 for more pins on a processor.  Then later they decided to not only go with the larger pin count, but bump from an 8 bit AVR to an ARM CM4F adding several dollars.  That sort of inconsistency drives me nuts.  "Every dollar counts!"  Yes it does, so you need to justify the cost *rationally*.  
>

Does every dollar count in this case? If this is for solving short term
ventilator shortages, should development speed not count more than dollars?

-- 
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail)

A national debt, if it is not excessive, will be to us a national blessing.
		-- Alexander Hamilton

Stef <stef33d@yahooi-n-v-a-l-i-d.com.invalid> wrote:
> On 2020-07-08 Rick C wrote in comp.arch.embedded:
> > It seems the UK has an abbreviated set of requirements for response to
> > COVID-19 and that is what they are mostly working with.  I forget the
> > name of it.
> 
> How long will this last? If you look at the new cases curves, the UK is well
> over the initial peak. https://www.nytimes.com/ has nice graphs of world
> cases on the front page.

I think the UK competition is over:
https://www.gov.uk/government/news/ventilator-challenge-hailed-a-success-as-uk-production-finishes

Demand from the rest of the world is a different, unclear, question...

Theo

On Wednesday, July 8, 2020 at 1:00:04 PM UTC-4, Stef wrote:
> On 2020-07-08 Rick C wrote in comp.arch.embedded:
> > On Wednesday, July 8, 2020 at 8:14:05 AM UTC-4, Stef wrote:
> >> On 2020-07-08 Rick C wrote in comp.arch.embedded:
> >> > Wow!  Designing in a group is strange.  I'm helping with the design of a ventilator and it is a unique experience.  We have people of many skill levels and talents, many I don't even know.
> >> 
> [...]
> 
> [Is there a way you could limit your line length to a more usenet friendly
> number?]

Not without copying messages to some other editor and then pasting back into Google groups.  I know there is a spec for this, but I've switched newsreaders several times over the years and I'm just tired of trying to find tools that last the test of time.  So I've settled for ease of use over strict adherence to the spec.  

Why is your newsreader intolerant of longer line length?  


> > It seems the UK has an abbreviated set of requirements for response to COVID-19 and that is what they are mostly working with.  I forget the name of it.  
> 
> How long will this last? If you look at the new cases curves, the UK is well
> over the initial peak. https://www.nytimes.com/ has nice graphs of world
> cases on the front page.

As was the case for much of the US... until they started to open things up again.  My crystal ball is not working so well these days.  Seems it needs a lubricant that is also used in making protective gear and is in short supply. 


> > They do get advice from time to time from professionals in the field.  My main concern is a direction they are currently going in where more and more valves, pressure relief, etc. is being added to the tubing right at the patient.  I'm not sure how it all will be held in place.  That and how they plan to get this through the approval process.  
> >
> 
> Sounds like they are moving to more and more features instead of sticking to a
> simple emergency use device. Feature creep is not uncommon when getting advise
> from several medical professionals. ;-)

Don't know about "more and more" or "feature creep".  They are responding to the issue that virtually every other piece of gear is doing the same limited job of ventilating an unconscious patient because it is easier.  Kennedy promoted the space race to the moon "not because it is easy, but because it is hard".  So why not take on the harder task of being not just more useful, but safer.  I don't recall all the terms, but when the patient is only assisted in breathing there is less risk of harm from use of the machine.  So this is not just a less often implemented feature, it is an important feature.  

In particular, it was in a conversation with a medical professional that this need was pointed out. 


> > I had a friend who worked on medical devices for a while before getting out.  He said they required that you "prove" your design will not hurt the patient in any way.  Hard to do with a ventilator.  
> >
> 
> That is not entirely true, it is more a risk/benefit analysis. Otherwise you
> could not even get a scalpel approved, it will always hurt the patient.

That is a bogus comparison.  So nothing to discuss there. 


> > I don't mind working on the team.  It's seeing them go in odd directions that bugs me.  This thing will probably cost hundreds of dollars in the final BOM without anything like profit or overhead added in.  People were arguing against adding $0.10 for more pins on a processor.  Then later they decided to not only go with the larger pin count, but bump from an 8 bit AVR to an ARM CM4F adding several dollars.  That sort of inconsistency drives me nuts.  "Every dollar counts!"  Yes it does, so you need to justify the cost *rationally*.  
> >
> 
> Does every dollar count in this case? If this is for solving short term
> ventilator shortages, should development speed not count more than dollars?

It's not about rushing to the finish line for sure.  None of the goals are clearly stated.  That's my point.  

We meet using video tools and Zoom seems to work the best.  I have various latency and bandwidth issues and was not able to participate yesterday.  That's really frustrating, especially when everyone is missing an important point.  Today we had a one on one to test Google Meet and compare to Zoom (Google was a second runner).  So I had a chance to make clear what is and is not a problem in the plumbing.  Understanding what I was saying he agreed so the plumbing next to the patient is less cumbersome now.  

Maybe more 1 on 1s will help. 

-- 

  Rick C.

  +- Get 1,000 miles of free Supercharging
  +- Tesla referral code - https://ts.la/richard11209