USB device bootloader: which method do you prefer?

I'm designing a small electronic gadget with a USB device port.

Now I'm thinking about a good method to upgrade the firmware through the 
USB. The process should be as simple as possible for the user.

I think there are three main method, each one with pros and cons.
In all of them, the user should only press a button while the device is 
connected to the host (so powered up). At startup, the switch is sensed 
and, if pressed, the bootloader enters or stop running the application.

CDC
The device is seen as a COM port from the host. IMHO this is the 
simplest method for the developer, but has many cons.
First of all, the user should be able to read the COM port number 
assigned by the host to the device.  However I don't know if it is 
really true. Is it possible to write a software that automatically 
detects if my USB CDC gadget is connected and automatically communicate 
to it without knowing the COM port number (or automatically detect it)?
Second, the user should install a driver when the device is attached (at 
lest in Windows, the driver is a .inf file only).
Third, a software must be installed on the host to communicate with the 
bootloader.

DFU
It seems USB defined a DFU standard for firmware upgrade through USB 
connection. If I understood well, even in this case the user should 
install a driver and should use a custom tool to manage the firmware 
upload process.

MSD
I think this is the simplest method for the user.
The host sees the device as a removable disk with a virtual filesystem 
in it (usually it is FAT12 that is the simplest to implement in the device).
The firmware upgrade process is only to drag the new file to the root 
folder.
The only issue I see here is how to report the result of firmware 
upgrade (success or error) to the user. He can only see a list of files.
I can think to create a virtual file, such as SUCCESS.TXT or ERROR.TXT 
or other similar names, when the firmware upgrade ends.  However the 
host OS could cache the content of the virtual disk, so hiding the new 
virtual files.

Maybe there are other methods. Which one do you prefer?

Am Mittwoch, 7. September 2016 09:57:45 UTC+2 schrieb pozz:
> I'm designing a small electronic gadget with a USB device port.
> 
> Now I'm thinking about a good method to upgrade the firmware through the 
> USB. The process should be as simple as possible for the user.
> CDC
> [..]
> DFU
> [..]
> MSD
> [..]
> Maybe there are other methods. Which one do you prefer?

Cypress uses HID for the PSoC devices, and we are using it, and it works ok.
No driver needed, some libs on Linux. 

Andreas

Il 07/09/2016 10:44, acd ha scritto:
> Am Mittwoch, 7. September 2016 09:57:45 UTC+2 schrieb pozz:
>> I'm designing a small electronic gadget with a USB device port.
>>
>> Now I'm thinking about a good method to upgrade the firmware through the
>> USB. The process should be as simple as possible for the user.
>> CDC
>> [..]
>> DFU
>> [..]
>> MSD
>> [..]
>> Maybe there are other methods. Which one do you prefer?
>
> Cypress uses HID for the PSoC devices, and we are using it, and it works ok.
> No driver needed, some libs on Linux.

Interesting. Anyway the user should install/launch a custom host 
software to upload the new firmware, shouldn't it?

How the host software recognize the correct HID device? Should the user 
select the device by a string (name) or some other identification mechanism?

I think many HID devices could be connected to the host (mouse, 
keyboard, joystick, ...)

Il 07/09/2016 11:44, pozz ha scritto:
> Il 07/09/2016 10:44, acd ha scritto:
>> Am Mittwoch, 7. September 2016 09:57:45 UTC+2 schrieb pozz:
>>> I'm designing a small electronic gadget with a USB device port.
>>>
>>> Now I'm thinking about a good method to upgrade the firmware through the
>>> USB. The process should be as simple as possible for the user.
>>> CDC
>>> [..]
>>> DFU
>>> [..]
>>> MSD
>>> [..]
>>> Maybe there are other methods. Which one do you prefer?
>>
>> Cypress uses HID for the PSoC devices, and we are using it, and it
>> works ok.
>> No driver needed, some libs on Linux.
>
> Interesting. Anyway the user should install/launch a custom host
> software to upload the new firmware, shouldn't it?
>
> How the host software recognize the correct HID device? Should the user
> select the device by a string (name) or some other identification
> mechanism?
>
> I think many HID devices could be connected to the host (mouse,
> keyboard, joystick, ...)

Maybe the host software searches for the exact VID/PID.

Op 07-Sep-16 om 9:57 AM schreef pozz:
> I'm designing a small electronic gadget with a USB device port.
>
> Now I'm thinking about a good method to upgrade the firmware through the
> USB. The process should be as simple as possible for the user.
>
> Maybe there are other methods. Which one do you prefer?

Make your device a HID and you don't need any drivers at the host side.

Wouter "Objects? No Thanks!" van Ooijen

Den onsdag den 7. september 2016 kl. 17.20.03 UTC+2 skrev Wouter van Ooijen:
> Op 07-Sep-16 om 9:57 AM schreef pozz:
> > I'm designing a small electronic gadget with a USB device port.
> >
> > Now I'm thinking about a good method to upgrade the firmware through the
> > USB. The process should be as simple as possible for the user.
> >
> > Maybe there are other methods. Which one do you prefer?
> 
> Make your device a HID and you don't need any drivers at the host side.
> 
> Wouter "Objects? No Thanks!" van Ooijen

you won't need drivers but you'll need some kind of program to send the firmware.  

pozz <pozzugno@gmail.com> wrote:
> MSD
> I think this is the simplest method for the user.
> The host sees the device as a removable disk with a virtual filesystem 
> in it (usually it is FAT12 that is the simplest to implement in the device).
> The firmware upgrade process is only to drag the new file to the root 
> folder.

Do you have enough memory to cache the file before writing, or do you have
to write as you go?  I'd want to check the hash/signature of the firmware
before committing to write it, in case the user does something stupid like
accidentally dragging the wrong file or not properly dismounting the disc
resulting in a partial write.

Such mistakes are less of a problem if you have a special flash program that
can display the appropriate skull and crossbones.

> The only issue I see here is how to report the result of firmware 
> upgrade (success or error) to the user. He can only see a list of files.
> I can think to create a virtual file, such as SUCCESS.TXT or ERROR.TXT 
> or other similar names, when the firmware upgrade ends.  However the 
> host OS could cache the content of the virtual disk, so hiding the new 
> virtual files.

Do you not have any out of band channel?  LEDs you can flash in a special
pattern or something?  You're right that the OS is likely to cache.  If not
I suppose you could hotplug something else - another MSD with title of
Success/Fail perhaps.

Theo

This decision is more based on what resources the target has.
It's also about the use cases for updates you expect to support.

Not to be snide, but I prefer to make things that work and don't
have to be updated. It's cheaper that way. I know this
isn't always possible but I bet it's more possible than you think.

Weasel out of iffy requirements with configuration, mostly. Test the
crap outta the thing - it's always cheaper  than you think.
If an "update" is shooting a new set of config options at the target,
your life is less interrupted.

If you need to make that look like an upgrade, then do that.

pozz wrote:
> I'm designing a small electronic gadget with a USB device port.
>
> Now I'm thinking about a good method to upgrade the firmware through the
> USB. The process should be as simple as possible for the user.
>
> I think there are three main method, each one with pros and cons.
> In all of them, the user should only press a button while the device is
> connected to the host (so powered up). At startup, the switch is sensed
> and, if pressed, the bootloader enters or stop running the application.
>
> CDC
> The device is seen as a COM port from the host. IMHO this is the
> simplest method for the developer, but has many cons.
> First of all, the user should be able to read the COM port number
> assigned by the host to the device.  However I don't know if it is
> really true. Is it possible to write a software that automatically
> detects if my USB CDC gadget is connected and automatically communicate
> to it without knowing the COM port number (or automatically detect it)?
> Second, the user should install a driver when the device is attached (at
> lest in Windows, the driver is a .inf file only).
> Third, a software must be installed on the host to communicate with the
> bootloader.
>

You can't reliably know which COM port will be assigned on the host
computer. There may be notification schemes in play ( usually a
popup happens on Windows at least ).

This being said, Arduinos use this and it works well.

http://stackoverflow.com/questions/4199083/detect-serial-port-insertion-removal

> DFU
> It seems USB defined a DFU standard for firmware upgrade through USB
> connection. If I understood well, even in this case the user should
> install a driver and should use a custom tool to manage the firmware
> upload process.
>

SFAIK, this may preclude using the USB port as a serial port. It may
not.

Ardunios use essentially an FDTI driver for both. It's a good
approach, IMO. But Ardunios don't have a file system and use what
appears to be a monolithic download image.

Cubie boards ( and I believe Beaglebone Black ) use this as an OTB port. 
You can then reflash ( soft-debrick ) the unit that way. I forget what 
happens when that goes wrong, you may need a JTAG/ODB type thing.

> MSD
> I think this is the simplest method for the user.
> The host sees the device as a removable disk with a virtual filesystem
> in it (usually it is FAT12 that is the simplest to implement in the
> device).
> The firmware upgrade process is only to drag the new file to the root
> folder.
> The only issue I see here is how to report the result of firmware
> upgrade (success or error) to the user. He can only see a list of files.
> I can think to create a virtual file, such as SUCCESS.TXT or ERROR.TXT
> or other similar names, when the firmware upgrade ends.  However the
> host OS could cache the content of the virtual disk, so hiding the new
> virtual files.
>

If you have the resources, sure. You still need a debrick strategy.

> Maybe there are other methods. Which one do you prefer?

Prefer? Something more like bootp. SLIP[1] the surly bonds
of serial ports.... then when you plug it in, it Just Updates.

[1] or PPP...

This may be a challenge for cases where users or techs have to update 
units in the field. But maybe you have an Official(tm) Li'l Upgrade Box
or something.

-- 
Les Cargill

On 9/7/2016 12:07 PM, Les Cargill wrote:
> Not to be snide, but I prefer to make things that work and don't
> have to be updated. It's cheaper that way. I know this
> isn't always possible but I bet it's more possible than you think.

+42

Too often, APPROACHING a product with the "upgrade path" foremost
in mind suggests a lack of confidence in:
- knowing what the customer wants
- knowing what the product should *be*
- your implementation
- some combination of the above

IME, "upgrades" are stressful experiences to most users.

The process is *rarely* "well documented" (i.e., instead of
saying "wait for the light to start blinking", you should
be saying "wait 7 seconds (not 6 and not 8) for the light
to start blinking".  *All* the details need to be nailed
down so the user doesn't wonder "how long should I wait?",
"is it doing what it *should* be doing?"  etc.

[I recently bricked a print server because "wait for the
device to reboot" REALLY meant "wait for the better portion
of a *minute* for the device to reboot".  AND, the device
had no provision for resuming an aborted update:  start it
and you are committed to finishing it!]

It typically takes a fair bit of time to update *a* single device:
- discover that an update is available
- decide if the update is worth installing
- discover if the update is *applicable* (version currently running?)
- download the update (make sure the download protocol alerts the
   user if the download was prematurely aborted!  make sure BINARY/ASCII
   mode doesn't impact the payload; etc.)
- set up the update environment (special cables?  disconnect peripherals?)
- perform the update
- verify the result (you *did* include a means whereby the user can
   identify the NEW version of the software installed, right?)
All assuming nothing goes wrong in the above.

If (when!) something goes wrong, is there a way to recover?  To restart
the upgrade (have you documented EVERYTHING the user needs to do in
order to restart?  E.g., my print server case required me to reset
the deviec so it would repeat the BOOTP discovery sequence)

If (when!) something goes wrong, is there a way to return to the previous
configuration?

If you later decide the upgrade was a "mistake" (you are not happy
with it -- or, it is buggier than your previous version, you've lost
some capability, etc.), is there a way to DOWNgrade?

Are all of your configuration choices seemlessly propagated forward to the
new implementation?

Repeat this for (potentially) *all* of your customers.  How many will
pick up a phone and call "support" somewhere in this process?  Or, send
an email asking for clarification/assistance?  How many will render their
devices useless?  How many of those will complain to you about your
"crappy software"?  What will all of this cost *you*?

And, more importantly, what will it cost *them*?  How eager will you be to
move yet another upgrade out to them?  How eager will they be to go through
this effort AGAIN?  When will they start thinking, "Why didn't the device
ALREADY have these features/capabilities WHEN I BOUGHT IT?"

> Weasel out of iffy requirements with configuration, mostly. Test the
> crap outta the thing - it's always cheaper  than you think.

If you're confident an upgrade will NOT be needed to fix a latent bug
in your code, then put your money where your mouth is:  challenge your
colleagues to see if they can find any bugs (in the product's use
*or* from an inspection of the sources).  I routinely bake goods for
friends/colleagues willing to spend some time hammering on something I've
decided is "done" -- a reasonably fair trade (a few hours of their time
vs. a few hours of me baking).  If someone finds something significant,
it would be a BARGAIN to treat them to a steak dinner -- "out".

If you find yourself "buying lots of dinners" -- or, "rationalizing
their (legitimate) comments away" -- then your process needs some serious
rethinking!

*Cherish* your customers' time -- because *they* do!  Treat them with
disdain and a competitor will lure them away in a heartbeat ("Our
product WORKS!")

Don Y <blockedofcourse@foo.invalid> wrote:
> Too often, APPROACHING a product with the "upgrade path" foremost
> in mind suggests a lack of confidence in:
> - knowing what the customer wants
> - knowing what the product should *be*
> - your implementation
> - some combination of the above
> 
> IME, "upgrades" are stressful experiences to most users.

Unfortunately, if your device has connectivity this doctrine no
longer applies.  In the brave new IoT world security threats will evolve and
vulnerabilities will happen.  While there are sensible mitigation
strategies, you can't plan for the discovery of vulnerabilities in advance.

Therefore you had better get used to an upgrade path and make it bomb-proof
because you will be using it.  Or if you aren't you're almost certainly
should be.  Unless you have a really good threat model as to why your gadget
isn't worth targeting (including using it as a springboard to attack other
things).

> The process is *rarely* "well documented" (i.e., instead of
> saying "wait for the light to start blinking", you should
> be saying "wait 7 seconds (not 6 and not 8) for the light
> to start blinking".  *All* the details need to be nailed
> down so the user doesn't wonder "how long should I wait?",
> "is it doing what it *should* be doing?"  etc.

To be honest UIs on these things are often terrible.  If you really want to
communicate with the user, put a display on it and tell them, don't try to
use Morse code.  But that costs money.

Theo