Looking for USB stack with permissive licence (mass storage)

Tim Wescott wrote:

> Have you checked with your chipset vendor to see if they've got support?

The chipsets are from ST. They do provide TCP/IP and USB stacks but
only as (expensive) options (high up-front cost + per-unit fees).
My bean-counters have deemed these costs unacceptable :-)

Which is why I've integrated the BSD-licensed lwip TCP/IP stack,
which is quite adequate. I was hoping to find the same quality
of software in an free USB stack.

> I see more and more chips coming out with on-board IP to run their 
> Ethernet or USB ports stashed away in ROM someplace.
> 
> Of course, you're then depending on code that's written by a group of 
> people who are viewed as an "expense" by company bean-counters.  But you 
> might luck out.

Open source is here to stay. Companies that do not provide source
code need to die a quick death.

Rob Gaddi wrote:

> Have you tried pinging your local Renesas FAE?  They might have app
> notey sorts of things with already working code.

Do you think I can contact Renesas for ST gear?

The only thing I want to do with the ST support team
involves voodoo dolls and large needles.

In article <jlk459$3n2$1@dont-email.me>, root@127.0.0.1 says...
> 
> Mark Borgerson wrote:
> 
> > Noob wrote:
> > 
> >> USB 2.0 "Hi-Speed" mode is 480 Mbit/s. My goal is 30 MB/s (240 Mbit/s)
> >> of "useful" bandwidth (i.e. considering actual data transferred only).
> >> IIUC, the overhead in the USB protocol is fairly significant.
> > 
> > That seems to be a pretty ambitious target.
> 
> I can live with 20-25 MB/s  ;-)

Might be lucky.
 
> > Where is all that data going to go (or come from)?
> 
> A recent 3.5" HDD in a USB2 enclosure (nowadays, 3.5" HDDs
> sport 64 MB of cache, and can sustain 50 MB/s over SATA)

SATA is 1.5Gbps max burst speed

SATA II is 3Gbps

SATA is point to point not daisy chain/star or other type of BUS
 
> > Do you want that rate in continuous transfer, or only in bursts?
> 
> The worst-case scenario (in terms of required performance)
> recording (writing) 4 HD programs (say 4x20 Mb/s) while
> playing one back (20 Mbit/s).
> 
> I.e. 100 Mbit/s (12.5 MB/s) of continuous bandwidth.
> Being able to sustain bursts at 30 MB/s would minimize
> time spent waiting for the bus.
> 
> > Given the overhead and the possibility
> > of other USB traffic, that seems a lot to ask.
> 
> What other USB traffic?

USB is basically a polled bus with time slices. Even with ine device you 
have root and management after enumeration etc.

Let alone talking to end points for each block transfer continuous or 
burst. And of course files system handling.
 
> >> Hmmm... The chipset we use provides its own USB controller, and the
> >> board layout has already been validated. I don't think it is an option,
> >> at this point, to add a new chip. Which is why I'm only looking for a
> >> software (USB2) stack. I only need support for the mass storage class
> >> and bulk transfers (don't know if the other transfer modes are required
> >> for normal operation).
> > 
> > Are you doing high-speed transfers now?  Do you get anything like
> > 30MB/second?
> 
> With the current stack, I've measured 18-20 MB/s sustained.
> But there are a lot of unnecessary copies because whoever
> ported the library was clueless about cached/uncached memory
> zones and DMA.

May well be best you get with USB.

Ever considered SATA( or II) or eSATA( or II) may be easier
Also allow any size mass storage, just have many filesystems to support.

-- 
Paul Carpenter          | paul@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/>    PC Services
<http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font
<http://www.gnuh8.org.uk/>  GNU H8 - compiler & Renesas H8/H8S/H8 Tiny
<http://www.badweb.org.uk/> For those web sites you hate

In article <jlk5cm$9jt$1@dont-email.me>, root@127.0.0.1 says...
> 
> Rob Gaddi wrote:
> 
> > Have you tried pinging your local Renesas FAE?  They might have app
> > notey sorts of things with already working code.
> 
> Do you think I can contact Renesas for ST gear?
> 
> The only thing I want to do with the ST support team
> involves voodoo dolls and large needles.

Well easy mistake to make as SH-4 architecture also sold by Renesas
(ex Hitachi division). May still be worth looking at their app notes
anyway.

-- 
Paul Carpenter          | paul@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/>    PC Services
<http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font
<http://www.gnuh8.org.uk/>  GNU H8 - compiler & Renesas H8/H8S/H8 Tiny
<http://www.badweb.org.uk/> For those web sites you hate

In article <jlk459$3n2$1@dont-email.me>, root@127.0.0.1 says...
> 
> Mark Borgerson wrote:
> 
> > Noob wrote:
> > 
> >> USB 2.0 "Hi-Speed" mode is 480 Mbit/s. My goal is 30 MB/s (240 Mbit/s)
> >> of "useful" bandwidth (i.e. considering actual data transferred only).
> >> IIUC, the overhead in the USB protocol is fairly significant.
> > 
> > That seems to be a pretty ambitious target.
> 
> I can live with 20-25 MB/s  ;-)
> 
> > Where is all that data going to go (or come from)?
> 
> A recent 3.5" HDD in a USB2 enclosure (nowadays, 3.5" HDDs
> sport 64 MB of cache, and can sustain 50 MB/s over SATA)

That's fine for SATA.  Have you measured the sustained throughput
with USB?
> 
> > Do you want that rate in continuous transfer, or only in bursts?
> 
> The worst-case scenario (in terms of required performance)
> recording (writing) 4 HD programs (say 4x20 Mb/s) while
> playing one back (20 Mbit/s).

That sounds like a good recipe for disk thrashing.  What kind
of file system are you planning to use on the HDD?  If it's
FAT32,  how will maintaining the FAT for 3 files being written
affect your throughput?   If it's a large disk, caching the 
whole FAT could use up a lot of your RAM.
> 
> I.e. 100 Mbit/s (12.5 MB/s) of continuous bandwidth.
> Being able to sustain bursts at 30 MB/s would minimize
> time spent waiting for the bus.
> 
> > Given the overhead and the possibility
> > of other USB traffic, that seems a lot to ask.
> 
> What other USB traffic?
Since you hadn't specified a dedicated HDD at that point, I had
to ask.
> 
> >> Hmmm... The chipset we use provides its own USB controller, and the
> >> board layout has already been validated. I don't think it is an option,
> >> at this point, to add a new chip. Which is why I'm only looking for a
> >> software (USB2) stack. I only need support for the mass storage class
> >> and bulk transfers (don't know if the other transfer modes are required
> >> for normal operation).
> > 
> > Are you doing high-speed transfers now?  Do you get anything like
> > 30MB/second?
> 
> With the current stack, I've measured 18-20 MB/s sustained.
> But there are a lot of unnecessary copies because whoever
> ported the library was clueless about cached/uncached memory
> zones and DMA.

To what degree is the throughput constrained by the HDD and file
system and not the unnecessary copies?
> 
> >> I find it strange that there are very good TCP/IP open source libraries,
> >> but that I can't find a portable, open source USB stack for embedded
> >> systems with "powerful" CPUs (powerful compared to micro-controllers,
> >> i.e. roughly equivalent to the first Pentium III variant Katmai.)
> > 
> > The variations in USB peripherals may be larger than that for Ethernet
> > controllers.  In addition,  Ethernet and TCP/IP are ancient when 
> > compared to USB 2.0.
> 
> :-(
And just when you thought USB 2.0 was good, there are now HDDs comming
out with USB 3.0 interfaces!   ;-)


Mark Borgerson

Paul wrote:

> Noob wrote:
> 
>> Rob Gaddi wrote:
>>
>>> Have you tried pinging your local Renesas FAE?  They might have app
>>> notey sorts of things with already working code.
>>
>> Do you think I can contact Renesas for ST gear?
>>
>> The only thing I want to do with the ST support team
>> involves voodoo dolls and large needles.
> 
> Well easy mistake to make as SH-4 architecture also sold by Renesas
> (ex Hitachi division). May still be worth looking at their app notes
> anyway.

A USB stack (even one limited to mass storage) is not what I would
call trivial software. I'm not sure what to expect from app notes.
Is it common for chip makers to provide full software stacks with
their app notes?

Regards.

Mark Borgerson wrote:

> Noob wrote:
> 
>> Mark Borgerson wrote:
>>
>>> Noob wrote:
>>>
>>>> USB 2.0 "Hi-Speed" mode is 480 Mbit/s. My goal is 30 MB/s (240 Mbit/s)
>>>> of "useful" bandwidth (i.e. considering actual data transferred only).
>>>> IIUC, the overhead in the USB protocol is fairly significant.
>>>
>>> That seems to be a pretty ambitious target.
>>
>> I can live with 20-25 MB/s  ;-)
>>
>>> Where is all that data going to go (or come from)?
>>
>> A recent 3.5" HDD in a USB2 enclosure (nowadays, 3.5" HDDs
>> sport 64 MB of cache, and can sustain 50 MB/s over SATA)
> 
> That's fine for SATA.  Have you measured the sustained throughput
> with USB?

"With the current stack, I've measured 18-20 MB/s sustained."
But this stack was compiled with gcc -O0, with message queues
re-implemented using interrupt masking (!!) instead of using
the OS implementation, with bounce buffers used everywhere,
and every dynamic memory block allocated from an uncached
memory pool.

> That sounds like a good recipe for disk thrashing.  What kind
> of file system are you planning to use on the HDD?  If it's
> FAT32,  how will maintaining the FAT for 3 files being written
> affect your throughput?   If it's a large disk, caching the 
> whole FAT could use up a lot of your RAM.

FAT sucks. We use ext2 with large blocks.

> To what degree is the throughput constrained by the HDD and file
> system and not the unnecessary copies?

Seriously? Modern HDDs can sustain 100+ MB/s sequential R/W on the
outer tracks, 50+ MB/s on the inner tracks. Whatever is the bottle
neck, it's not the HDD.

> And just when you thought USB 2.0 was good, there are now HDDs comming
> out with USB 3.0 interfaces!   ;-)

I don't think the higher bandwidth of USB 3 brings anything
to the table for a PVR product. Can one plug a USB3 enclosure
into a USB2 port?

Regards.

In comp.arch.embedded,
Noob <root@127.0.0.1> wrote:
> Paul wrote:
>
>> Noob wrote:
>> 
>>> Rob Gaddi wrote:
>>>
>>>> Have you tried pinging your local Renesas FAE?  They might have app
>>>> notey sorts of things with already working code.
>>>
>>> Do you think I can contact Renesas for ST gear?
>>>
>>> The only thing I want to do with the ST support team
>>> involves voodoo dolls and large needles.
>> 
>> Well easy mistake to make as SH-4 architecture also sold by Renesas
>> (ex Hitachi division). May still be worth looking at their app notes
>> anyway.
>
> A USB stack (even one limited to mass storage) is not what I would
> call trivial software. I'm not sure what to expect from app notes.
> Is it common for chip makers to provide full software stacks with
> their app notes?

I don't know if its 'common', but Atmel did supply a working mass storage
example for their SAM7S controllers.

-- 
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail)

The number of licorice gumballs you get out of a gumball machine
increases in direct proportion to how much you hate licorice.

On 04/16/2012 01:41 PM, Stef wrote:
> In comp.arch.embedded,
> Noob<root@127.0.0.1>  wrote:
>> Paul wrote:
>>
>>> Noob wrote:
>>>
>>>> Rob Gaddi wrote:
>>>>
>>>>> Have you tried pinging your local Renesas FAE?  They might have app
>>>>> notey sorts of things with already working code.
>>>>
>>>> Do you think I can contact Renesas for ST gear?
>>>>
>>>> The only thing I want to do with the ST support team
>>>> involves voodoo dolls and large needles.
>>>
>>> Well easy mistake to make as SH-4 architecture also sold by Renesas
>>> (ex Hitachi division). May still be worth looking at their app notes
>>> anyway.
>>
>> A USB stack (even one limited to mass storage) is not what I would
>> call trivial software. I'm not sure what to expect from app notes.
>> Is it common for chip makers to provide full software stacks with
>> their app notes?
>
> I don't know if its 'common', but Atmel did supply a working mass storage
> example for their SAM7S controllers.
>

If you strip it down to a point where the host only works with a single 
mass storage device, without a hub, and you don't worry about current 
consumption, suspend mode, or isochronous endpoints, it's not that hard 
to make a USB stack. The app notes I've seen are not much more than this.

In comp.arch.embedded,
Arlet Ottens <usenet+5@c-scape.nl> wrote:
> On 04/16/2012 01:41 PM, Stef wrote:
>> In comp.arch.embedded,
>> Noob<root@127.0.0.1>  wrote:
>>> Paul wrote:
>>>
>>>> Noob wrote:
>>>>
>>>>> Rob Gaddi wrote:
>>>>>
>>>>>> Have you tried pinging your local Renesas FAE?  They might have app
>>>>>> notey sorts of things with already working code.
>>>>>
>>>>> Do you think I can contact Renesas for ST gear?
>>>>>
>>>>> The only thing I want to do with the ST support team
>>>>> involves voodoo dolls and large needles.
>>>>
>>>> Well easy mistake to make as SH-4 architecture also sold by Renesas
>>>> (ex Hitachi division). May still be worth looking at their app notes
>>>> anyway.
>>>
>>> A USB stack (even one limited to mass storage) is not what I would
>>> call trivial software. I'm not sure what to expect from app notes.
>>> Is it common for chip makers to provide full software stacks with
>>> their app notes?
>>
>> I don't know if its 'common', but Atmel did supply a working mass storage
>> example for their SAM7S controllers.
>>
>
> If you strip it down to a point where the host only works with a single 
> mass storage device, without a hub, and you don't worry about current 
> consumption, suspend mode, or isochronous endpoints, it's not that hard 
> to make a USB stack. The app notes I've seen are not much more than this.

Whoops, I think missed hte 'host' part somewhere. The Atmel example is
device only, sorry.

-- 
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail)

A computer without COBOL and Fortran is like a piece of chocolate cake
without ketchup and mustard.