EmbeddedRelated.com
Forums

SD/MMC card on an sam7-ex256

Started by axl_dudu March 2, 2008
Hi Paul et al,

This leads me to something else that's been bothering me for some time.

I notice that SD FAT libs make some big assumptions about FAT type.
If I format FAT16A on my 512 MB card, (eg) EFSL starts using FAT32, although it should use FAT16.
Anyone know of any docs somewhere that update on how to make a better decision ?
(apart from fatgen103.doc). I'm pondering this too especially since MS dropped the lower limit for
FAT32 to 64 MB ....
(Perhaps looking at the FsInfo record if there, and then go FAT32 or some such ?)

I always thought that simply looking at the FATXX string in the MBR is not the way to do it,
but seems to me that stupidly looking at disk size isn't the right way either.

XP does figure out with larger cards whether I've formatted them in FAT16 or FAT32 on my own HW.

Secondly, I'd like to come up with a more precise way to look at the MBR of cards to see whether
they're SFD or FDISK. I've resorted to looking for the "FAT" string in LBA #0 (since a partitioned
card won't have that there).
Should I access the first partition (if it's there), fetch the 1st partition's boot record and
examine whether the signature 0xAA55 is at [511] ?
If it's not there - wouldn't that be a more reliable way to then conclude that this is an SFD card
(ie. NO partitions) ?

Best Regards,
Kris

-----Original Message-----
From: A... [mailto:A...] On Behalf Of Paul Curtis
Sent: Monday, 10 March 2008 3:48 AM
To: A...
Subject: RE: [AT91SAM] Re: Reading extra partitions on SD Cards as Volumes on XP

Hi Tudor,

> Hello Paul,
> sorry for asking, but I didn't understood one thing from what you
> wrote. Is MAC OS able to recognize different partitions on a sigle SD
> card?

Yes, and it's able to partition them too.

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
Hi Kris,

> This leads me to something else that's been bothering me for some time.
>
> I notice that SD FAT libs make some big assumptions about FAT type.
> If I format FAT16A on my 512 MB card, (eg) EFSL starts using FAT32,
> although it should use FAT16.
> Anyone know of any docs somewhere that update on how to make a better
> decision ?
> (apart from fatgen103.doc).

That is *the* specification to use. Microsoft advise that, for best interoperability, the number of clusters in a volume should not be near the thresholds for FAT12/FAT16/FAT32 because they've seen broken implementations of FAT. From my experience, the only thing that matters is the number of clusters in a volume to determine the volume type. NOTE: even Microsoft mis-document the threshold conditions in their tools!

> I'm pondering this too especially since MS dropped the lower limit for
> FAT32 to 64 MB ....

I'm not sure what you're getting at here.

> (Perhaps looking at the FsInfo record if there, and then go FAT32 or
> some such ?)

Number of clusters. Always.

> I always thought that simply looking at the FATXX string in the MBR is
> not the way to do it,

Absolutely not!

> but seems to me that stupidly looking at disk size isn't the right way
> either.

Correct, disk size has nothing to do with it. You can have FAT16 all the way to 2G. There is a method for determining FAT12/16/32 given the number of sectors, but that's for selecting the best configuration, not the configuration that will actually be used.

> XP does figure out with larger cards whether I've formatted them in
> FAT16 or FAT32 on my own HW.
> Secondly, I'd like to come up with a more precise way to look at the
> MBR of cards to see whether
> they're SFD or FDISK. I've resorted to looking for the "FAT" string in
> LBA #0 (since a partitioned card won't have that there).

Nope, don't do that. Again, superfloppy and those with a partition map are slightly different but might "look the same". There are some good links about for decoding extended partitions and the like, but not real firm documentation on how partition maps and the MBR all jive together. When Vista write the new GUID-based partition tables it also fakes up an MBR. Mac OS can do this stuff too.

> Should I access the first partition (if it's there), fetch the 1st
> partition's boot record and
> examine whether the signature 0xAA55 is at [511] ?
> If it's not there - wouldn't that be a more reliable way to then
> conclude that this is an SFD card
> (ie. NO partitions) ?

Here is an extract from my simple volume mount, which is *not* the full mount code for disks. It just tries to mount a volume as a superfloppy or the first active partition (note: not all the code is shown here):

 // If there is a jump then this is a drive without partitions such as a floppy
// or a superfloppy (SD/MMC card, USB flash drive, and so on).
if ((vol->sector_data.bytes[0x000] == 0xEB && vol->sector_data.bytes[0x002] == 0x90) ||
vol->sector_data.bytes[0x000] == 0xE9)
{
partition_start_sector = 0;
}
else
{
// Select first partition.
if (vol->sector_data.bytes[0x1BF])
{
// Partition 1 is a valid partition.
partition_start_sector = xtl_fat_read_uint32(vol->sector_data.bytes + 0x1C6);
}

Rgds,

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors



Hi Kris,

> Hi Paul,
>
> Thanks for clarifying, that confirms my observations :-)
>
> > Whatever it's doing, the partitions don't show up. I don't think
> anybody
> > has been able to do this on Windows.
>
> Can you define 'partitions don't show up' better ?
> Disk Management does show my (extra) partitions, but it doesn't see the
> FAT I put on it.

Only the first active partition of a SD card is mounted and available under
Windows evcen if there are multiple partitions on a disk. Mac OS X shows
you all the partitions and you can use them all.

> While on this subject, can you confirm that each boot record for a
> partition does _indeed_ reside
> at LBA start of partition + the CHS sector offset of 63 ?

No idea what you're getting at here Kris. I don't recognize the 63. I just
use the tables...

> IOW I mean, let's say an SD card is partitioned exactly in 2 equal
> parts (SD card size/2).
> If I have this correct, the Boot record for partition #2 would be at
> LBA (SD card size/2) + 63
> and its LBA sector size (at [0x1CE].sectorsize in MBR) would be LBA
> ((SD card size/2) - 63).
> Correct ?

Why? Again, the 63 isn't all that illuminating for me.

> In any case, it's still handy to partition. One of my test cards is a
> Transcend 512 MB and it's woefully slow. For quicker testing I just
allocate like say 1/4 of that
> etc.

If you read the information returned by the card you find a fair amount of
information about the internals of the card. The newer SDHC cards are rated
and will attain a sustained transfer rate of up to 6MB/second for class 6
cards. That's not slow. If your file system isn't very intelligent, I/O
will be slow.

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
Hi Paul,

> number of clusters in a volume to determine the volume type.
> NOTE: even Microsoft mis-document he threshold conditions in their tools!

Yep, I've noticed that ! If you apply their 'magic formula' (fatgen103.doc) at times you can get
inconsistencies of 1-2 sectors for the threshold... MS themselves can't even get it right.
It's funny actually, quoting fatgen : "Just don't wonder how the formula works, it just does.."
Well, it doesn't ALWAYS.

>> I'm pondering this too especially since MS dropped the lower limit for
>> FAT32 to 64 MB ....
> I'm not sure what you're getting at here.

Well, I meant that a 64 MB volume does NOT necessarily mean that it'll be FAT16 (anymore).
I can format 64 MB cards as FAT32 or FAT

> Here is an extract from my simple volume mount, which is *not* the full mount code for disks.
> It just tries to mount a volume as a superfloppy or the first active partition (note: not all
> the code is shown here):

Right, I see - so you're looking at the actual bootcode (jmp) at [0] (0x0090FEEB)
I haven't come across the 2nd possibility yet where MBR[0] = 0xE9 ? What does that cover ?
(I'm happy to go figure it out though).

This is already a great help, thanks !
(I'm doing this to learn more about it)
Best Regards,
Kris

-----Original Message-----
From: A... [mailto:A...] On Behalf Of Paul Curtis
Sent: Monday, 10 March 2008 9:44 AM
To: A...
Subject: RE: [AT91SAM] Re: Reading extra partitions on SD Cards as Volumes on XP

Hi Kris,

> This leads me to something else that's been bothering me for some time.
>
> I notice that SD FAT libs make some big assumptions about FAT type.
> If I format FAT16A on my 512 MB card, (eg) EFSL starts using FAT32,
> although it should use FAT16.
> Anyone know of any docs somewhere that update on how to make a better
> decision ?
> (apart from fatgen103.doc).

That is *the* specification to use. Microsoft advise that, for best interoperability, the number
of clusters in a volume should not be near the thresholds for FAT12/FAT16/FAT32 because they've
seen broken implementations of FAT. From my experience, the only thing that matters is the number
of clusters in a volume to determine the volume type. NOTE: even Microsoft mis-document the
threshold conditions in their tools!
Hi Paul,

> No idea what you're getting at here Kris. I don't recognize the 63. I just
> use the tables...

I'm referring to the standard offset for a partition table (Max # of sectors in CHS I believe).
This is for my own formatting, so I'm creating those tables.

I'll try reword this :

Let's say I have a disk with 20,000 (decimal) sectors on it. I format into 2 Volumes, each 10,000
sectors.

So,
Partition table #1 would have :
[8] : # of sectors (LBA) preceding its partition (absolute) = 63
[C] : # of sectors in the partition = (10,000 - 63)

Partition table #2 would have :
[8] : # of sectors (LBA) preceding its partition (absolute) = 10,000
[C] : # of sectors in the partition = (10,000 - 63)

In this case, where do I store the Boot record for Partition #2 ?
At absolute sector # 10,000 - Right ??? I initially would have thought that the boot record for
partition # 2 would have been at absolute sector # 10,063 !!

Perhaps it's confusing because I'm simplifying it with the partition table offset.
(if that makes sense :-)
I haven't covered CHS for clarity.

> If you read the information returned by the card you find a fair amount of
> information about the internals of the card.

That I don't get.
What if I decide to only allocate say 1/4 of the card (by using one partition and not SFD),
leaving 3/4 unallocated ? What good is the CIS then ????
If I based my decision what FAT to use that way, all hell will break loose.

> Only the first active partition of a SD card is mounted and available under
> Windows evcen if there are multiple partitions on a disk. Mac OS X shows
> you all the partitions and you can use them all.

I don't use Mac (besides I hate those dumb Mac & PC ads with the guy from Die Hard 4.0, they're
full of crap). I love Die Hard 4.0 however.. :-)

Shame Windows doesn't readily read multiple partitions.
Kinda makes me wonder whether I should still bother then with partitioning in my format code ?
Oh well, I'm still having fun and learning anyhow.
Best Regards,
Kris

-----Original Message-----
From: A... [mailto:A...] On Behalf Of Paul Curtis
Sent: Monday, 10 March 2008 9:50 AM
To: A...
Subject: RE: [AT91SAM] Reading extra partitions on SD Cards as Volumes on XP
Hi,

I have played with some FAT libs form sam7x and 7s part and I finaly used
the chanfat. I use it principally in fat16 but it is suppose to support the
fat12/16/32 also.

It is only for your information. I have played with the esfl but without
many success.

http://elm-chan.org/fsw/ff/00index_e.html

regards

Jonathan

-----Message d'origine-----
De: A... [mailto:A...] De la part de
microbit
Envoy 9 mars 2008 18:01
: A...
Objet: RE: [AT91SAM] Re: Reading extra partitions on SD Cards as Volumes on
XP

Hi Paul et al,

This leads me to something else that's been bothering me for some time.

I notice that SD FAT libs make some big assumptions about FAT type.
If I format FAT16A on my 512 MB card, (eg) EFSL starts using FAT32, although
it should use FAT16.
Anyone know of any docs somewhere that update on how to make a better
decision ?
(apart from fatgen103.doc). I'm pondering this too especially since MS
dropped the lower limit for
FAT32 to 64 MB ....
(Perhaps looking at the FsInfo record if there, and then go FAT32 or some
such ?)

I always thought that simply looking at the FATXX string in the MBR is not
the way to do it,
but seems to me that stupidly looking at disk size isn't the right way
either.

XP does figure out with larger cards whether I've formatted them in FAT16 or
FAT32 on my own HW.

Secondly, I'd like to come up with a more precise way to look at the MBR of
cards to see whether
they're SFD or FDISK. I've resorted to looking for the "FAT" string in LBA
#0 (since a partitioned
card won't have that there).
Should I access the first partition (if it's there), fetch the 1st
partition's boot record and
examine whether the signature 0xAA55 is at [511] ?
If it's not there - wouldn't that be a more reliable way to then conclude
that this is an SFD card
(ie. NO partitions) ?

Best Regards,
Kris


-----Original Message-----
From: A... [mailto:A...] On Behalf Of
Paul Curtis
Sent: Monday, 10 March 2008 3:48 AM
To: A...
Subject: RE: [AT91SAM] Re: Reading extra partitions on SD Cards as Volumes
on XP

Hi Tudor,

> Hello Paul,
> sorry for asking, but I didn't understood one thing from what you
> wrote. Is MAC OS able to recognize different partitions on a sigle SD
> card?

Yes, and it's able to partition them too.

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors



Kris,

> > If you read the information returned by the card you find a fair
> amount of information about the internals of the card.
>
> That I don't get.

Shame. ;-)

> What if I decide to only allocate say 1/4 of the card (by using one
> partition and not SFD),
> leaving 3/4 unallocated ? What good is the CIS then ????
> If I based my decision what FAT to use that way, all hell will break
> loose.

This is not what I mean. The card can tell you about things that will make
it go faster at the interface using multi-block writes. I believe this is
required, in effect, for all these digital cameras to take snappy
photos--there's no way to make things go fast on a highly fragmented SD card
(consider every other sector in use). For streaming large amounts of data
to a file you really need a good filing system that knows a little about the
capability of the media it's driving.

> > Only the first active partition of a SD card is mounted and available
> under
> > Windows evcen if there are multiple partitions on a disk. Mac OS X
> shows
> > you all the partitions and you can use them all.
>
> I don't use Mac (besides I hate those dumb Mac & PC ads with the guy
> from Die Hard 4.0, they're
> full of crap). I love Die Hard 4.0 however.. :-)
>
> Shame Windows doesn't readily read multiple partitions.
> Kinda makes me wonder whether I should still bother then with
> partitioning in my format code ?
> Oh well, I'm still having fun and learning anyhow.

I can't see the need to partition anything on an embedded device--format a
partition, possibly, but not do that actual partitioning.

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for ARM, MSP430, AVR, MAXQ, and now Cortex-M3 processors
Kris,

> > number of clusters in a volume to determine the volume type.
> > NOTE: even Microsoft mis-document he threshold conditions in their
> tools!
>
> Yep, I've noticed that ! If you apply their 'magic formula'
> (fatgen103.doc) at times you can get
> inconsistencies of 1-2 sectors for the threshold... MS themselves can't
> even get it right.
> It's funny actually, quoting fatgen : "Just don't wonder how the
> formula works, it just does.."
> Well, it doesn't ALWAYS.

Well, it seems to work for me, at least I cannot see where it would fail.

> >> I'm pondering this too especially since MS dropped the lower limit
> for
> >> FAT32 to 64 MB ....
> > I'm not sure what you're getting at here.
>
> Well, I meant that a 64 MB volume does NOT necessarily mean that it'll
> be FAT16 (anymore). I can format 64 MB cards as FAT32 or FAT

You can use FAT32 on anything (within reason).

> > Here is an extract from my simple volume mount, which is *not* the
> full mount code for disks.
> > It just tries to mount a volume as a superfloppy or the first active
> partition (note: not all
> > the code is shown here):
>
> Right, I see - so you're looking at the actual bootcode (jmp) at [0]
> (0x0090FEEB)
> I haven't come across the 2nd possibility yet where MBR[0] = 0xE9 ?
> What does that cover ? (I'm happy to go figure it out though).

See fatgen103 p. 9.

-- Paul.
Well, XP _can_ treat USB card readers as 'Local'...

Just recently Bill Knight gave me a great URL for doing just exactly what I was after.
I tried it and it works like a charm on my cardreader :
http://portableapps.com/node/7181

As Bill pointed out, you don't need to access the registry though :
The info the web page says to get from the registry is also on the 'Details' tab of the disk
device properties dialog. It's actually really simple.
(of course you can always roll back the driver)

You can create/delete Partitions from XP to your heart's content.
So I picked up a bit more on my formatting function.
I'm still testing more, and cleaning up a bit but the func can now :
- Create a single SD as SFD (ie. no partition tables, just MBR).
- Create a single SD as a single Partition using the reported Card Size to adjust the partition.
- Create a single SD as a single Partition. Or with 2,3 or 4 partitions.
- Have arbitrary sizes specified for each Partition.
- Give each SFD or Partition an arbitrary Serial Number 32 bit.
- Give each SFD or Partition an arbitrary Volume label.
- Format in FAT12, FAT16(16 bit sectors), FAT16(32 bit sectors), FAT32(LBA or CHS)
- Set the decision threshold for FAT16->FAT32 (default 65525, but I set it at 64MB so I can create
FAT16 partitions where 32 bit sector values are used.

I've tested a fair bit and it seems stable now.
FAT types can be in any order, as long as they pass the plausibility check.
(You can partition from XP but you're limited with choice of Volume sizes)
Once I have it completely abstracted, so you can 1-2-3 link it with the rest of EFSL (and tested
for that), I'll upload it.

There's still snags too in the IO manager that haven't been fixed in the latest versions ie. the
error handling for sector writes in IOMan is really buggered up.
It can affect proper flushing of the cache.

When I have a gap again will tinker more there too.

Best Regards,
Kris

-----Original Message-----
From: A... [mailto:A...] On Behalf Of axl_dudu
Sent: Monday, 10 March 2008 2:45 AM
To: A...
Subject: [AT91SAM] Re: Reading extra partitions on SD Cards as Volumes on XP

Hi Kris,
Good ideea to use more partitions on a single SD card. I don't see
any reason for Windows to not recognize them.
If you find anything, please let me know.

Tudor

--- In A..., "microbit" wrote:
=======================================================================Groups related to AT91SAM
=======================================================================
msp430 (131 common members)
http://groups.yahoo.com/group/msp430?v=1&t=ipt&ch=email&pub=groups&slktr0&sec=recg
Computers & Internet/Hardware: Welcome to the MSP430 User's Group! The purpose of...

avrclub (118 common members)
http://groups.yahoo.com/group/avrclub?v=1&t=ipt&ch=email&pub=groups&slktr1&sec=recg
Microprocessors/Microcontrollers: Atmel AVR Microcontroller discussion group!

AVR-Chat (109 common members)
http://groups.yahoo.com/group/AVR-Chat?v=1&t=ipt&ch=email&pub=groups&slktr2&sec=recg
Microprocessors/Microcontrollers: A place for Atmel AVR Microcontroller users to sha...

e-embedded (82 common members)
http://groups.yahoo.com/group/e-embedded?v=1&t=ipt&ch=email&pub=groups&slktr3&sec=recg
Internet/Internet Appliances: Open-membership mailing list for embedded system d...

rabbit-semi (71 common members)
http://groups.yahoo.com/group/rabbit-semi?v=1&t=ipt&ch=email&pub=groups&slktr4&sec=recg
Microprocessors/Microcontrollers: This is a user group for folks designing and progr...
GREAT! Thanks a lot Kris!

Tudor
--- In A..., "microbit" wrote:
>
> Well, XP _can_ treat USB card readers as 'Local'...
>
> Just recently Bill Knight gave me a great URL for doing just
exactly what I was after.
> I tried it and it works like a charm on my cardreader :
> http://portableapps.com/node/7181
>
> As Bill pointed out, you don't need to access the registry though :
> The info the web page says to get from the registry is also on
the 'Details' tab of the disk
> device properties dialog. It's actually really simple.
> (of course you can always roll back the driver)
>
> You can create/delete Partitions from XP to your heart's content.
> So I picked up a bit more on my formatting function.
> I'm still testing more, and cleaning up a bit but the func can now :
> - Create a single SD as SFD (ie. no partition tables, just MBR).
> - Create a single SD as a single Partition using the reported Card
Size to adjust the partition.
> - Create a single SD as a single Partition. Or with 2,3 or 4
partitions.
> - Have arbitrary sizes specified for each Partition.
> - Give each SFD or Partition an arbitrary Serial Number 32 bit.
> - Give each SFD or Partition an arbitrary Volume label.
> - Format in FAT12, FAT16(16 bit sectors), FAT16(32 bit sectors),
FAT32(LBA or CHS)
> - Set the decision threshold for FAT16->FAT32 (default 65525, but I
set it at 64MB so I can create
> FAT16 partitions where 32 bit sector values are used.
>
> I've tested a fair bit and it seems stable now.
> FAT types can be in any order, as long as they pass the
plausibility check.
> (You can partition from XP but you're limited with choice of Volume
sizes)
> Once I have it completely abstracted, so you can 1-2-3 link it with
the rest of EFSL (and tested
> for that), I'll upload it.
>
> There's still snags too in the IO manager that haven't been fixed
in the latest versions ie. the
> error handling for sector writes in IOMan is really buggered up.
> It can affect proper flushing of the cache.
>
> When I have a gap again will tinker more there too.
>
> Best Regards,
> Kris
>
>
> -----Original Message-----
> From: A... [mailto:A...] On
Behalf Of axl_dudu
> Sent: Monday, 10 March 2008 2:45 AM
> To: A...
> Subject: [AT91SAM] Re: Reading extra partitions on SD Cards as
Volumes on XP
>
> Hi Kris,
> Good ideea to use more partitions on a single SD card. I don't see
> any reason for Windows to not recognize them.
> If you find anything, please let me know.
>
> Tudor
>
> --- In A..., "microbit" wrote:
>