Forums

Bootloading from SD card

Started by eeboy November 16, 2009
I want to enable my device to replace the contents of flash by reading in a
binary file from the SD card. The catch is that I don't want to implement
FAT in my bootloader. I want it to be as small in size and as simple as can
be. 

The vast majority of the users will be running windows. The binary file
(representing the new flash contents) does not have to be placed on the SD
card by conventional methods (dragging the downloaded file from hard disk
to SD in explorer). I can force the user to place the file onto the SD card
using the application used to configure the device. Another constraint is
that the SD card format (FAT) must be left alone as the user needs to read
and write files that the device has generated. So, how can I write this
file to the SD card from a windows based machine such that my device can
read the firmware without knowing anything of FAT (yet FAT is still present
on the card)?

Hopefully the problem makes sense... :)	   
					
---------------------------------------		
This message was sent using the comp.arch.embedded web interface on
http://www.EmbeddedRelated.com
On Nov 16, 6:57=A0am, "eeboy" <ja...@jasonorsborn.com> wrote:
> I want to enable my device to replace the contents of flash by reading in=
a
> binary file from the SD card. The catch is that I don't want to implement > FAT in my bootloader. I want it to be as small in size and as simple as c=
an
> be. > > The vast majority of the users will be running windows. The binary file > (representing the new flash contents) does not have to be placed on the S=
D
> card by conventional methods (dragging the downloaded file from hard disk > to SD in explorer). I can force the user to place the file onto the SD ca=
rd
> using the application used to configure the device. Another constraint is > that the SD card format (FAT) must be left alone as the user needs to rea=
d
> and write files that the device has generated. So, how can I write this > file to the SD card from a windows based machine such that my device can > read the firmware without knowing anything of FAT (yet FAT is still prese=
nt
> on the card)? > > Hopefully the problem makes sense... :) =A0 =A0 > > --------------------------------------- =A0 =A0 =A0 =A0 > This message was sent using the comp.arch.embedded web interface onhttp:/=
/www.EmbeddedRelated.com you need to use my NoFAT(TM) method 1) format SD 2) place a file with 4MByte HEADER + payload to the card 3) in embedded system a) read 1 sector at 4MB abs offset, check header calc offset to payload b) read payload header should include signature and "counter" that gives offset to payload this method has been used in many systems already Antti
On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote:

> I want to enable my device to replace the contents of flash by reading > in a binary file from the SD card. The catch is that I don't want to > implement FAT in my bootloader. I want it to be as small in size and as > simple as can be. > > The vast majority of the users will be running windows. The binary file > (representing the new flash contents) does not have to be placed on the > SD card by conventional methods (dragging the downloaded file from hard > disk to SD in explorer). I can force the user to place the file onto the > SD card using the application used to configure the device. Another > constraint is that the SD card format (FAT) must be left alone as the > user needs to read and write files that the device has generated. So, > how can I write this file to the SD card from a windows based machine > such that my device can read the firmware without knowing anything of > FAT (yet FAT is still present on the card)? > > Hopefully the problem makes sense... :) > > --------------------------------------- This message was sent using the > comp.arch.embedded web interface on http://www.EmbeddedRelated.com
AFAIK a FAT filesystem is pretty lightweight; make it so it only understands a single file with a specific name and size and it should be even lighter weight yet (I think this is what Antti was getting at). That'll leave it compatible with Windows, and may be good enough for what you want. I suggest you build something into your system that makes the file in the SD card unique to your board -- a filename, a checksum method, something. You don't want to burn the right code for the wrong product into your board! -- www.wescottdesign.com
Tim Wescott wrote:
> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: > >> I want to enable my device to replace the contents of flash by reading >> in a binary file from the SD card. The catch is that I don't want to >> implement FAT in my bootloader. I want it to be as small in size and as >> simple as can be. >> >> The vast majority of the users will be running windows. The binary file >> (representing the new flash contents) does not have to be placed on the >> SD card by conventional methods (dragging the downloaded file from hard >> disk to SD in explorer). I can force the user to place the file onto the >> SD card using the application used to configure the device. Another >> constraint is that the SD card format (FAT) must be left alone as the >> user needs to read and write files that the device has generated. So, >> how can I write this file to the SD card from a windows based machine >> such that my device can read the firmware without knowing anything of >> FAT (yet FAT is still present on the card)? >> >> Hopefully the problem makes sense... :) >> >> --------------------------------------- This message was sent using the >> comp.arch.embedded web interface on http://www.EmbeddedRelated.com > > AFAIK a FAT filesystem is pretty lightweight; make it so it only > understands a single file with a specific name and size and it should be > even lighter weight yet (I think this is what Antti was getting at). > > That'll leave it compatible with Windows, and may be good enough for what > you want. > > I suggest you build something into your system that makes the file in the > SD card unique to your board -- a filename, a checksum method, > something. You don't want to burn the right code for the wrong product > into your board! >
A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD card to create a larger RESEVERED AREA. See link for details: http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR The RESEVERED AREA area is always at the beginning of the "disk area" and is just consecutive sectors. FAT understands this RESEVERED AREA and will just jump over it. This area is also hidden from FAT users. don

eeboy wrote:

> I want to enable my device to replace the contents of flash by reading in a > binary file from the SD card. The catch is that I don't want to implement > FAT in my bootloader. I want it to be as small in size and as simple as can > be. > > The vast majority of the users will be running windows. The binary file > (representing the new flash contents) does not have to be placed on the SD > card by conventional methods (dragging the downloaded file from hard disk > to SD in explorer). I can force the user to place the file onto the SD card > using the application used to configure the device. Another constraint is > that the SD card format (FAT) must be left alone as the user needs to read > and write files that the device has generated. So, how can I write this > file to the SD card from a windows based machine such that my device can > read the firmware without knowing anything of FAT (yet FAT is still present > on the card)? > > Hopefully the problem makes sense... :)
Huh? Isn't it how all operating systems are booting? 1. Create a contigious file on FAT (attributes "system", "unmovable") 2. Put your data into this file. 3. Read this file not as a file, but as a sequence of sectors. The bootloader only has to know what sector is the beginning of the file. If you don't want to parse directories, put the reference to the starting sector into boot sector. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
In article <2sadnVdv24iqHpzWnZ2dnUVZ_tudnZ2d@forethought.net>, don <don> wrote:
>Tim Wescott wrote: >> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: >> >>> I want to enable my device to replace the contents of flash by reading >>> in a binary file from the SD card. The catch is that I don't want to >>> implement FAT in my bootloader. I want it to be as small in size and as >>> simple as can be. >>> >>> The vast majority of the users will be running windows. The binary file >>> (representing the new flash contents) does not have to be placed on the >>> SD card by conventional methods (dragging the downloaded file from hard >>> disk to SD in explorer). I can force the user to place the file onto the >>> SD card using the application used to configure the device. Another >>> constraint is that the SD card format (FAT) must be left alone as the >>> user needs to read and write files that the device has generated. So, >>> how can I write this file to the SD card from a windows based machine >>> such that my device can read the firmware without knowing anything of >>> FAT (yet FAT is still present on the card)? >> >A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD card >to create a larger RESEVERED AREA. > >See link for details: >http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR > >The RESEVERED AREA area is always at the beginning of the "disk area" >and is just consecutive sectors. > >FAT understands this RESEVERED AREA and will just jump over it. >This area is also hidden from FAT users.
Along the same lines, you can create two partitions on the memory card: one is FAT and the other one is your own personal space. Set the partition ID byte to something that Windows doesn't recognize and it will completely ignore your partition and will never attempt to read or wite to it. Your bootloader only needs to know how to read a partition table, which is fairly trivial.
On Nov 16, 8:59=A0pm, t...@nospam.com (Tom) wrote:
> In article <2sadnVdv24iqHpzWnZ2dnUVZ_tudn...@forethought.net>, don <don> =
wrote:
> >Tim Wescott wrote: > >> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: > > >>> I want to enable my device to replace the contents of flash by readin=
g
> >>> in a binary file from the SD card. The catch is that I don't want to > >>> implement FAT in my bootloader. I want it to be as small in size and =
as
> >>> simple as can be. > > >>> The vast majority of the users will be running windows. The binary fi=
le
> >>> (representing the new flash contents) does not have to be placed on t=
he
> >>> SD card by conventional methods (dragging the downloaded file from ha=
rd
> >>> disk to SD in explorer). I can force the user to place the file onto =
the
> >>> SD card using the application used to configure the device. Another > >>> constraint is that the SD card format (FAT) must be left alone as the > >>> user needs to read and write files that the device has generated. So, > >>> how can I write this file to the SD card from a windows based machine > >>> such that my device can read the firmware without knowing anything of > >>> FAT (yet FAT is still present on the card)? > > >A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD card > >to create a larger RESEVERED AREA. > > >See link for details: > >http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR > > >The RESEVERED AREA area is always at the beginning of the "disk area" > >and is just consecutive sectors. > > >FAT understands this RESEVERED AREA and will just jump over it. > >This area is also hidden from FAT users. > > Along the same lines, you can create two partitions on the memory card: o=
ne is
> FAT and the other one is your own personal space. Set the partition ID by=
te to
> something that Windows doesn't recognize and it will completely ignore yo=
ur
> partition and will never attempt to read or wite to it. Your bootloader o=
nly
> needs to know how to read a partition table, which is fairly trivial.
I think you missed eeboy's intent. He wants the card to be not only FAT formatted, but the file to be accessed by the bootloader is to be written to the card using windows normal file access although it can be from a program rather than through the windows GUI. I get what he is doing. In fact, I might want to do something similar myself. I would like to be able to read an SD or MMC card using an FPGA without a CPU. As long as the access is very simple, this should not be a problem. The idea of a single file on the card with direct access by knowing the sector offset rather than going through the FAT file system would make this not so hard in an FPGA. Antti is suggesting a method of writing a file of known content and then searching it from the target system to measure the offset I believe. Rick
On Wed, 18 Nov 2009 10:05:34 -0800 (PST), rickman <gnuarm@gmail.com>
wrote:

>On Nov 16, 8:59&#2013266080;pm, t...@nospam.com (Tom) wrote: >> In article <2sadnVdv24iqHpzWnZ2dnUVZ_tudn...@forethought.net>, don <don> wrote: >> >Tim Wescott wrote: >> >> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: >> >> >>> I want to enable my device to replace the contents of flash by reading >> >>> in a binary file from the SD card. The catch is that I don't want to >> >>> implement FAT in my bootloader. I want it to be as small in size and as >> >>> simple as can be. >> >> >>> The vast majority of the users will be running windows. The binary file >> >>> (representing the new flash contents) does not have to be placed on the >> >>> SD card by conventional methods (dragging the downloaded file from hard >> >>> disk to SD in explorer). I can force the user to place the file onto the >> >>> SD card using the application used to configure the device. Another >> >>> constraint is that the SD card format (FAT) must be left alone as the >> >>> user needs to read and write files that the device has generated. So, >> >>> how can I write this file to the SD card from a windows based machine >> >>> such that my device can read the firmware without knowing anything of >> >>> FAT (yet FAT is still present on the card)? >> >> >A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD card >> >to create a larger RESEVERED AREA. >> >> >See link for details: >> >http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR >> >> >The RESEVERED AREA area is always at the beginning of the "disk area" >> >and is just consecutive sectors. >> >> >FAT understands this RESEVERED AREA and will just jump over it. >> >This area is also hidden from FAT users. >> >> Along the same lines, you can create two partitions on the memory card: one is >> FAT and the other one is your own personal space. Set the partition ID byte to >> something that Windows doesn't recognize and it will completely ignore your >> partition and will never attempt to read or wite to it. Your bootloader only >> needs to know how to read a partition table, which is fairly trivial. > >I think you missed eeboy's intent. He wants the card to be not only >FAT formatted, but the file to be accessed by the bootloader is to be >written to the card using windows normal file access although it can >be from a program rather than through the windows GUI. I get what he >is doing. In fact, I might want to do something similar myself. I >would like to be able to read an SD or MMC card using an FPGA without >a CPU. As long as the access is very simple, this should not be a >problem. The idea of a single file on the card with direct access by >knowing the sector offset rather than going through the FAT file >system would make this not so hard in an FPGA. > >Antti is suggesting a method of writing a file of known content and >then searching it from the target system to measure the offset I >believe.
Yes, that could be done. Write the file to an otherwise empty filesystem. It should then start at cluster 2, and hopefully sequentially thereafter. Might have to write it on a older DOS system to get that to work. When updating the file, OVERWRITE the existing file, do not delete and then write it. Create it initially large as it will ever need to be. The firmware loader should be able to start at a known sector address and read sectors in order. -- ArarghMail911 at [drop the 'http://www.' from ->] http://www.arargh.com BCET Basic Compiler Page: http://www.arargh.com/basic/index.html To reply by email, remove the extra stuff from the reply address.
>On Nov 16, 8:59=A0pm, t...@nospam.com (Tom) wrote: >> In article <2sadnVdv24iqHpzWnZ2dnUVZ_tudn...@forethought.net>, don <don>
=
>wrote: >> >Tim Wescott wrote: >> >> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: >> >> >>> I want to enable my device to replace the contents of flash by
readin=
>g >> >>> in a binary file from the SD card. The catch is that I don't want
to
>> >>> implement FAT in my bootloader. I want it to be as small in size and
=
>as >> >>> simple as can be. >> >> >>> The vast majority of the users will be running windows. The binary
fi=
>le >> >>> (representing the new flash contents) does not have to be placed on
t=
>he >> >>> SD card by conventional methods (dragging the downloaded file from
ha=
>rd >> >>> disk to SD in explorer). I can force the user to place the file onto
=
>the >> >>> SD card using the application used to configure the device. Another >> >>> constraint is that the SD card format (FAT) must be left alone as
the
>> >>> user needs to read and write files that the device has generated.
So,
>> >>> how can I write this file to the SD card from a windows based
machine
>> >>> such that my device can read the firmware without knowing anything
of
>> >>> FAT (yet FAT is still present on the card)? >> >> >A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD
card
>> >to create a larger RESEVERED AREA. >> >> >See link for details: >> >http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR >> >> >The RESEVERED AREA area is always at the beginning of the "disk area" >> >and is just consecutive sectors. >> >> >FAT understands this RESEVERED AREA and will just jump over it. >> >This area is also hidden from FAT users. >> >> Along the same lines, you can create two partitions on the memory card:
o=
>ne is >> FAT and the other one is your own personal space. Set the partition ID
by=
>te to >> something that Windows doesn't recognize and it will completely ignore
yo=
>ur >> partition and will never attempt to read or wite to it. Your bootloader
o=
>nly >> needs to know how to read a partition table, which is fairly trivial. > >I think you missed eeboy's intent. He wants the card to be not only >FAT formatted, but the file to be accessed by the bootloader is to be >written to the card using windows normal file access although it can >be from a program rather than through the windows GUI. I get what he >is doing. In fact, I might want to do something similar myself. I >would like to be able to read an SD or MMC card using an FPGA without >a CPU. As long as the access is very simple, this should not be a >problem. The idea of a single file on the card with direct access by >knowing the sector offset rather than going through the FAT file >system would make this not so hard in an FPGA. > >Antti is suggesting a method of writing a file of known content and >then searching it from the target system to measure the offset I >believe. > >Rick >
Rick is correct. Given the ideas presented above it sounds as if I would need to take control of the formatting as well but I am not sure if I want to get that restrictive. I may just have to implement a portion of FAT in my bootloader so that I can seek out a particular filename/filesize. The problem is that the user can (and likely will at some point) format the SD card via Windows Explorer and use it in the device. So, I can't rely on any specific format. The device it is used in would then generate a large number of files (up to 16GB) as well. Now, say the user wants to upgrade firmware. So, they place the SD card in a card reader and poke a button in my application that prepares the SD card. Given the suggestions, I would have to copy off existing data temporarily, format the card using one of the suggestions above, write the binary file to a known offset and finally replace the data on the card. --------------------------------------- This message was sent using the comp.arch.embedded web interface on http://www.EmbeddedRelated.com
On Wed, 18 Nov 2009 18:07:16 -0600, "eeboy" <jason@jasonorsborn.com>
wrote:

>>On Nov 16, 8:59=A0pm, t...@nospam.com (Tom) wrote: >>> In article <2sadnVdv24iqHpzWnZ2dnUVZ_tudn...@forethought.net>, don <don> >= >>wrote: >>> >Tim Wescott wrote: >>> >> On Sun, 15 Nov 2009 22:57:56 -0600, eeboy wrote: >>> >>> >>> I want to enable my device to replace the contents of flash by >readin= >>g >>> >>> in a binary file from the SD card. The catch is that I don't want >to >>> >>> implement FAT in my bootloader. I want it to be as small in size and >= >>as >>> >>> simple as can be. >>> >>> >>> The vast majority of the users will be running windows. The binary >fi= >>le >>> >>> (representing the new flash contents) does not have to be placed on >t= >>he >>> >>> SD card by conventional methods (dragging the downloaded file from >ha= >>rd >>> >>> disk to SD in explorer). I can force the user to place the file onto >= >>the >>> >>> SD card using the application used to configure the device. Another >>> >>> constraint is that the SD card format (FAT) must be left alone as >the >>> >>> user needs to read and write files that the device has generated. >So, >>> >>> how can I write this file to the SD card from a windows based >machine >>> >>> such that my device can read the firmware without knowing anything >of >>> >>> FAT (yet FAT is still present on the card)? >>> >>> >A simple way is to modify the boot sector (Cluster 0) on teh MMC/SD >card >>> >to create a larger RESEVERED AREA. >>> >>> >See link for details: >>> >http://www.compuphase.com/mbr_fat.htm#BOOTSECTOR >>> >>> >The RESEVERED AREA area is always at the beginning of the "disk area" >>> >and is just consecutive sectors. >>> >>> >FAT understands this RESEVERED AREA and will just jump over it. >>> >This area is also hidden from FAT users. >>> >>> Along the same lines, you can create two partitions on the memory card: >o= >>ne is >>> FAT and the other one is your own personal space. Set the partition ID >by= >>te to >>> something that Windows doesn't recognize and it will completely ignore >yo= >>ur >>> partition and will never attempt to read or wite to it. Your bootloader >o= >>nly >>> needs to know how to read a partition table, which is fairly trivial. >> >>I think you missed eeboy's intent. He wants the card to be not only >>FAT formatted, but the file to be accessed by the bootloader is to be >>written to the card using windows normal file access although it can >>be from a program rather than through the windows GUI. I get what he >>is doing. In fact, I might want to do something similar myself. I >>would like to be able to read an SD or MMC card using an FPGA without >>a CPU. As long as the access is very simple, this should not be a >>problem. The idea of a single file on the card with direct access by >>knowing the sector offset rather than going through the FAT file >>system would make this not so hard in an FPGA. >> >>Antti is suggesting a method of writing a file of known content and >>then searching it from the target system to measure the offset I >>believe. >> >>Rick >> > >Rick is correct. Given the ideas presented above it sounds as if I would >need to take control of the formatting as well but I am not sure if I want >to get that restrictive. I may just have to implement a portion of FAT in >my bootloader so that I can seek out a particular filename/filesize.
It's not all that difficult for FAT12 or FAT16 if you restrict your file to the root directory. I wrote a routine that searches the root directory for a specific file, and reads it into memory via the FAT. I think it took less than 512 bytes on an x86, not sure, but definitely less than 1024 bytes. The boot sector is only 2 sectors long -- non standard disk format - but I presume your routine would be in ROM or flash so my non-standard format is not an issue. FAT32 would be a somewhat larger headache. :-) There are at least two web sites that have disassemblies of x86 bootsectors for FAT12, FAT16, and FAT32. The code may not be useful, but the logic would be.
>The problem is that the user can (and likely will at some point) format the
More than likely. :-)
>SD card via Windows Explorer and use it in the device. So, I can't rely on >any specific format. The device it is used in would then generate a large >number of files (up to 16GB) as well. Now, say the user wants to upgrade >firmware. So, they place the SD card in a card reader and poke a button in >my application that prepares the SD card. Given the suggestions, I would >have to copy off existing data temporarily, format the card using one of >the suggestions above, write the binary file to a known offset and finally >replace the data on the card.
If it's the first thing you write to a newly formatted drive, it would be. Except that windows may not use sequential allocation units. -- ArarghMail911 at [drop the 'http://www.' from ->] http://www.arargh.com BCET Basic Compiler Page: http://www.arargh.com/basic/index.html To reply by email, remove the extra stuff from the reply address.