Our LPC2478 device acts as a USB mass storage device. As well as the USB
host (Windows in this case) being able to read and write files on it,
the device itself can also manipulate the files on the "USB drive".
However if I delete or create files on the "USB drive" from the device
rather than from Windows, Windows does not know about the changes until
I unplug the USB and reconnect it. Does anyone know if this is normal,
or if my mass storage implementation is missing some sort of "refresh"
command?
--
Tim Mitchell
USB Mass storage question
Started by ●March 23, 2009
Reply by ●March 23, 20092009-03-23
Tim Mitchell wrote:
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
>
There is no 'refresh' command where the storage device (the LPC/SD card
from Window's point of view,
and Windows/LPC-driver/SD card from the LPC's point of view) tells the
'other' side what happened.
Cluster aware filesystems can do this but they need to be synchronized
in some ways.
While you could technically do this since the LPC is aware of all
modifications, Windows does not
understand that the underlying filesystem is magically modified and it
does not know what has been
modified, nor does it see any reason to write all modified buffers
immediately back to the storage device,
so both LPC and Windows see an incomplete view of the current filesystem.
The result is a sever filesystem corruption.
The only things here that'll work is read-only from ALL connected
devices. No one is allowed to write
as the other participants won't expect anything to be modified.
Harald
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
>
There is no 'refresh' command where the storage device (the LPC/SD card
from Window's point of view,
and Windows/LPC-driver/SD card from the LPC's point of view) tells the
'other' side what happened.
Cluster aware filesystems can do this but they need to be synchronized
in some ways.
While you could technically do this since the LPC is aware of all
modifications, Windows does not
understand that the underlying filesystem is magically modified and it
does not know what has been
modified, nor does it see any reason to write all modified buffers
immediately back to the storage device,
so both LPC and Windows see an incomplete view of the current filesystem.
The result is a sever filesystem corruption.
The only things here that'll work is read-only from ALL connected
devices. No one is allowed to write
as the other participants won't expect anything to be modified.
Harald
Reply by ●March 23, 20092009-03-23
Hi Tim,
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
As far as I know, and from my experimentation, you can't do what you want to
do without informing the host (Windows) that the drive might have changed.
You can do this by generating a UNIT ATTENTION condition using a
not-read-to-ready change plus medium-may-have-changed. You can then the
next sense key can say "medium not present" and then the next
test-unit-ready would say indicate that it is.
This is all a bit hypothetical as I haven't tried it yet.
In essence you want to tell the Windows end that the USB media has gone away
and come back.
I'm not sure this is a brilliant idea, having two ends reading/writing the
same thing. Windows delays some writes, so I'm not sure how you can provide
interlocks.
--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
As far as I know, and from my experimentation, you can't do what you want to
do without informing the host (Windows) that the drive might have changed.
You can do this by generating a UNIT ATTENTION condition using a
not-read-to-ready change plus medium-may-have-changed. You can then the
next sense key can say "medium not present" and then the next
test-unit-ready would say indicate that it is.
This is all a bit hypothetical as I haven't tried it yet.
In essence you want to tell the Windows end that the USB media has gone away
and come back.
I'm not sure this is a brilliant idea, having two ends reading/writing the
same thing. Windows delays some writes, so I'm not sure how you can provide
interlocks.
--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
Reply by ●March 23, 20092009-03-23
Paul Curtis wrote:
> You can do this by generating a UNIT ATTENTION condition using a
> not-read-to-ready change plus medium-may-have-changed. You can then the
> next sense key can say "medium not present" and then the next
> test-unit-ready would say indicate that it is.
>
>
That actually might work:
Make Windows believe this is a removable medium. Then Windows does not
do write buffering.
Then, when the LPC wants to make a change on the SD card, tell Windows
the medium was removed,
modify the filesystem, then tell Windows the medium is back. Since
Windows does not know it's the same
medium, it'll assume nothing and re-read everything it needs.
When Windows opens a file, it'll lock the medium. Do not allow the LPC
to access it until Windows releases it.
What you do here is strictly serializing access, and while I think the
performance will be slow, it might just work.
I fail to see the purpose here though. If you already have Windows, you
already have a hard disk (or similar) with lots
of space to write to. And it might be easier to let the LPC via the
Windows machine write to a file on the hard disk
via file system operations (not on block level).
Harald
> You can do this by generating a UNIT ATTENTION condition using a
> not-read-to-ready change plus medium-may-have-changed. You can then the
> next sense key can say "medium not present" and then the next
> test-unit-ready would say indicate that it is.
>
>
That actually might work:
Make Windows believe this is a removable medium. Then Windows does not
do write buffering.
Then, when the LPC wants to make a change on the SD card, tell Windows
the medium was removed,
modify the filesystem, then tell Windows the medium is back. Since
Windows does not know it's the same
medium, it'll assume nothing and re-read everything it needs.
When Windows opens a file, it'll lock the medium. Do not allow the LPC
to access it until Windows releases it.
What you do here is strictly serializing access, and while I think the
performance will be slow, it might just work.
I fail to see the purpose here though. If you already have Windows, you
already have a hard disk (or similar) with lots
of space to write to. And it might be easier to let the LPC via the
Windows machine write to a file on the hard disk
via file system operations (not on block level).
Harald
Reply by ●March 23, 20092009-03-23
Hi Tim,
I can imagine you can pass information from the device to windows and back
if:
1. You precreate (before allowing USB connection) big enough file(s), and
pass the information as a content of this/those fixed files.
2. There is your own proprietary application running on windows which
accesses the files with CreateFile(...., FILE_FLAG_NO_BUFFERING |
FILE_FLAG_WRITE_THROUGH, ...)
3. Noone changes the sizes of the files and noone else writes to the
filesystem.
With regards,
Jan
----- Original Message -----
From: "Tim Mitchell"
To:
Sent: Monday, March 23, 2009 12:47 PM
Subject: [lpc2000] USB Mass storage question
Our LPC2478 device acts as a USB mass storage device. As well as the USB
host (Windows in this case) being able to read and write files on it,
the device itself can also manipulate the files on the "USB drive".
However if I delete or create files on the "USB drive" from the device
rather than from Windows, Windows does not know about the changes until
I unplug the USB and reconnect it. Does anyone know if this is normal,
or if my mass storage implementation is missing some sort of "refresh"
command?
--
Tim Mitchell
I can imagine you can pass information from the device to windows and back
if:
1. You precreate (before allowing USB connection) big enough file(s), and
pass the information as a content of this/those fixed files.
2. There is your own proprietary application running on windows which
accesses the files with CreateFile(...., FILE_FLAG_NO_BUFFERING |
FILE_FLAG_WRITE_THROUGH, ...)
3. Noone changes the sizes of the files and noone else writes to the
filesystem.
With regards,
Jan
----- Original Message -----
From: "Tim Mitchell"
To:
Sent: Monday, March 23, 2009 12:47 PM
Subject: [lpc2000] USB Mass storage question
Our LPC2478 device acts as a USB mass storage device. As well as the USB
host (Windows in this case) being able to read and write files on it,
the device itself can also manipulate the files on the "USB drive".
However if I delete or create files on the "USB drive" from the device
rather than from Windows, Windows does not know about the changes until
I unplug the USB and reconnect it. Does anyone know if this is normal,
or if my mass storage implementation is missing some sort of "refresh"
command?
--
Tim Mitchell
Reply by ●March 23, 20092009-03-23
Paul Curtis schrieb:
> I'm not sure this is a brilliant idea, having two ends reading/writing the
> same thing. Windows delays some writes, so I'm not sure how you can provide
> interlocks.
One may tell Windows not to buffer writes (immediate removal or
something like this) but there are still many situations where the FS
will be corrupted.
HCC embedded has a solution for this, but I don't know how it works.
--
42Bastian
Note: SPAM-only account, direct mail to bs42@...
> I'm not sure this is a brilliant idea, having two ends reading/writing the
> same thing. Windows delays some writes, so I'm not sure how you can provide
> interlocks.
One may tell Windows not to buffer writes (immediate removal or
something like this) but there are still many situations where the FS
will be corrupted.
HCC embedded has a solution for this, but I don't know how it works.
--
42Bastian
Note: SPAM-only account, direct mail to bs42@...
Reply by ●March 23, 20092009-03-23
Hi,
windows will send "prevent/allow medium removal" requests before
changing the media. Using Paul:s method plus these requests it might
work. Though hosts not behaving this way will corrupt the file-system. I
would worry about embedded host implementations.
Foltos
Paul Curtis wrote:
> Hi Tim,
>
>
>> Our LPC2478 device acts as a USB mass storage device. As well as the USB
>> host (Windows in this case) being able to read and write files on it,
>> the device itself can also manipulate the files on the "USB drive".
>>
>> However if I delete or create files on the "USB drive" from the device
>> rather than from Windows, Windows does not know about the changes until
>> I unplug the USB and reconnect it. Does anyone know if this is normal,
>> or if my mass storage implementation is missing some sort of "refresh"
>> command?
>>
>
> As far as I know, and from my experimentation, you can't do what you want to
> do without informing the host (Windows) that the drive might have changed.
>
> You can do this by generating a UNIT ATTENTION condition using a
> not-read-to-ready change plus medium-may-have-changed. You can then the
> next sense key can say "medium not present" and then the next
> test-unit-ready would say indicate that it is.
>
> This is all a bit hypothetical as I haven't tried it yet.
>
> In essence you want to tell the Windows end that the USB media has gone away
> and come back.
>
> I'm not sure this is a brilliant idea, having two ends reading/writing the
> same thing. Windows delays some writes, so I'm not sure how you can provide
> interlocks.
>
> --
> Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
> CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
>
>
windows will send "prevent/allow medium removal" requests before
changing the media. Using Paul:s method plus these requests it might
work. Though hosts not behaving this way will corrupt the file-system. I
would worry about embedded host implementations.
Foltos
Paul Curtis wrote:
> Hi Tim,
>
>
>> Our LPC2478 device acts as a USB mass storage device. As well as the USB
>> host (Windows in this case) being able to read and write files on it,
>> the device itself can also manipulate the files on the "USB drive".
>>
>> However if I delete or create files on the "USB drive" from the device
>> rather than from Windows, Windows does not know about the changes until
>> I unplug the USB and reconnect it. Does anyone know if this is normal,
>> or if my mass storage implementation is missing some sort of "refresh"
>> command?
>>
>
> As far as I know, and from my experimentation, you can't do what you want to
> do without informing the host (Windows) that the drive might have changed.
>
> You can do this by generating a UNIT ATTENTION condition using a
> not-read-to-ready change plus medium-may-have-changed. You can then the
> next sense key can say "medium not present" and then the next
> test-unit-ready would say indicate that it is.
>
> This is all a bit hypothetical as I haven't tried it yet.
>
> In essence you want to tell the Windows end that the USB media has gone away
> and come back.
>
> I'm not sure this is a brilliant idea, having two ends reading/writing the
> same thing. Windows delays some writes, so I'm not sure how you can provide
> interlocks.
>
> --
> Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
> CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
>
>
Reply by ●March 23, 20092009-03-23
----Original Message----
> Paul Curtis wrote:
> >
> > I'm not sure this is a brilliant idea, having two ends
> > reading/writing
> > the same thing. Windows delays some writes, so I'm not
> > sure how you
> > can provide interlocks.
> >
OK, thanks for all the info, I've learned some stuff I didn't know.
It would be acceptable to only allow the LPC to access the "disk" when
USB is not enumerated so I think I will just do it that way and avoid
any potential pitfalls. I just wondered if there was a way to do it that
I didn't know about.
This all explains why my digital camera locks out all the user controls
while connected by USB.
--
Tim Mitchell
> Paul Curtis wrote:
> >
> > I'm not sure this is a brilliant idea, having two ends
> > reading/writing
> > the same thing. Windows delays some writes, so I'm not
> > sure how you
> > can provide interlocks.
> >
OK, thanks for all the info, I've learned some stuff I didn't know.
It would be acceptable to only allow the LPC to access the "disk" when
USB is not enumerated so I think I will just do it that way and avoid
any potential pitfalls. I just wondered if there was a way to do it that
I didn't know about.
This all explains why my digital camera locks out all the user controls
while connected by USB.
--
Tim Mitchell
Reply by ●March 23, 20092009-03-23
> Posted by: "Tim Mitchell"
>
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
To tell the host that things have changed, get the TUR SCSI
command (Test Unit Ready) to return SenseNotReady Key,
ASC:, ASC. Part of the trouble is that Windows caches the
whole of the FAT.
When the host writes to the disk, you must tell your FAT file
system that its sector cache is now invalid.
This mechanism appears to be safe for Mac OSX, Linux and
Windows.
Stephen
--
Stephen Pelc, s...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
>
> Our LPC2478 device acts as a USB mass storage device. As well as the USB
> host (Windows in this case) being able to read and write files on it,
> the device itself can also manipulate the files on the "USB drive".
>
> However if I delete or create files on the "USB drive" from the device
> rather than from Windows, Windows does not know about the changes until
> I unplug the USB and reconnect it. Does anyone know if this is normal,
> or if my mass storage implementation is missing some sort of "refresh"
> command?
To tell the host that things have changed, get the TUR SCSI
command (Test Unit Ready) to return SenseNotReady Key,
ASC:, ASC. Part of the trouble is that Windows caches the
whole of the FAT.
When the host writes to the disk, you must tell your FAT file
system that its sector cache is now invalid.
This mechanism appears to be safe for Mac OSX, Linux and
Windows.
Stephen
--
Stephen Pelc, s...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
Reply by ●March 25, 20092009-03-25
> It would be acceptable to only allow the LPC to
access the "disk" when
> USB is not enumerated so I think I will just do it that way and avoid
> any potential pitfalls.
Actually, I don't think there any reliable options. The USB mass storage device implements a set of low level commands (usually SCSI commands) to access the media directly at a very low level. The actual FAT file system code is remote and it will most likely have a cached copy of the FAT on the host machine.
When you also mount the media locally on the client side, you will then have another FAT file system that probably has its own cached copy of the FAT. What happens if they both try to update the FAT at the same time?
Basically you have two file systems operating on the same media with no interlocks, no coordination and no knowledge of each other. The fact is that you are very lucky that you only had files that are not visible. Complete corruption of the FAT format is also possible.
Greg
> USB is not enumerated so I think I will just do it that way and avoid
> any potential pitfalls.
Actually, I don't think there any reliable options. The USB mass storage device implements a set of low level commands (usually SCSI commands) to access the media directly at a very low level. The actual FAT file system code is remote and it will most likely have a cached copy of the FAT on the host machine.
When you also mount the media locally on the client side, you will then have another FAT file system that probably has its own cached copy of the FAT. What happens if they both try to update the FAT at the same time?
Basically you have two file systems operating on the same media with no interlocks, no coordination and no knowledge of each other. The fact is that you are very lucky that you only had files that are not visible. Complete corruption of the FAT format is also possible.
Greg
