Compact flash transfer rate (w/microcontroller)

Markus Zingg wrote:
> >> I.e. on a Rabbit 2000 I achived ~450KB/sec "netto" write speed towards
> >> a FAT 16 filesystem.
> >
> >Wow.. that's great! Considering that you also update the fat, the
> >"gross" (raw sectors) write rate is probably even higher!
> >Did you perform  "single" sector writes or "multiple"?
>
> Single sector writes are just fine. Thats especially true if you
> consider the fact that memory is costly on small systems. Todays CF
> cards will anyways be magnitudes faster than your embedded controller

Depends on the CF.  One major CF card controller is a re-programmed ARM
controller.  Until flash can be programmed faster than 100x, there is
no point speeding up the controller anyway.

> (asuming your are not playing with a 400Mhz XScale or such :-) ) so

FYI, Lastest Xscale is 624 MHz.

> ...

> Ohh yesss definately. Again, especially the code writing and reading a
> sector is VERY important. I got the best results by also using A10
> that said the so called but totally missleading "memory mapped mode".
> There you can use special block move operations your controller might
> support (as it's the case with the R2000 i.e.). Note thought that the
> difference is not all that big. 
> 

> The C versus assembly story is of course different when it comes to
> implementing FAT. I never ever would consider implementing FAT in
> assembly these days. That's an area where 'C' is much more apropriate.

So a good idea may be implementing low-level sector access routines is 
asm, and using them in C for handling FAT16...


> 
>>And what about the impact of:
>>- memory buffers
> 
> 
> You must buffer wisely. 
> 
I just need one file, and as you suggest I might pre-map it with fixed 
size, for example filling it with 0, and then changing only the data 
sectors...(with no overhead updating the fat)
Shall the buffering unit be one sector or one cluster?


But definitely, as I have only to append new data to existing one in the 
flash.. what happens when I issue to the flash a WRITE SECTOR command?

After writing the command register and the other ones, the flash expects 
512 bytes to be written (with A2 A1 A0 set to 0 0 0 , right?)

But, what happens if I write for example only 20 bytes, then I wait some 
time by letting my cpu doing other things, and then write other 60 
bytes, then I wait some time doing some calculations and finally write 
the remaining 512-20-60 bytes?

I mean... can I use the CF as a single sector buffer? Or just have I to 
write all the 512B of a sector in a single shot with no interruptions?




> 
> I'm one of those crazy people who are faster in writing their own
> library than understanding someone elses code :-) Seriousely, I had to
> write all of it on my own for various reasons. 

yeah! I'm one of those guys too! (...sometimes :-) )




> However, if you only are after logging data, you can prealocate the
> file in a first simple step, then simply write consecutive sectors
> during the real logging. If you do this cleverly, you really don't
> have to have much really FAT specific code hence you might save lots
> of time.

Good hint, tnx.



 > Don't expect to be able to port existing say Linux FAT Code
> to an 8 bitter though. 

Didn't even think about this! :-)




> Code that deals with sector IO should be readily available. If you
> really are in the dare need of this, I can help you out. 
> 
Thank you, I already have found something, I'll first check this out.



Markus, thanks for the wise advices and the good hints, it was a really 
helpful discussion. Bye!
Aldo

Markus Zingg wrote:
> >> I.e. on a Rabbit 2000 I achived ~450KB/sec "netto" write speed towards
> >> a FAT 16 filesystem.
> ...
> Code that deals with sector IO should be readily available. If you
> really are in the dare need of this, I can help you out.

I have a GPL ATA/ATAPI implementation via GPIO of Microchip PIC18. Its
inner loop peaks at a bit under 2MB/sec (PIO) on a 40MHz (ext clock)
PIC: http://www.telegraphics.com.au/svn/picide/trunk (Subversion).

> 
> HTH
> 
> Markus

>So a good idea may be implementing low-level sector access routines is 
>asm, and using them in C for handling FAT16...

That's what I tried to tell yes.

>I just need one file, and as you suggest I might pre-map it with fixed 
>size, for example filling it with 0, and then changing only the data 
>sectors...(with no overhead updating the fat)
>Shall the buffering unit be one sector or one cluster?

Obviousely just one sector since with your aproach there are no
clusters. I mean if you format the media and then create the file in
advance as the one and only file (asuming you do that external in a PC
or such) you can count on the fact that you have subsequent sectors.
That said you don't even have to read the fat at all to follow
clusters. You would have to guarantee though that never ever a user
inserts a CF into your device that was not prepared this way.
Otherwise you still need to read along the cluster chain and in doing
so you may want to spend a buffer for caching the FAT.

>But definitely, as I have only to append new data to existing one in the 
>flash.. what happens when I issue to the flash a WRITE SECTOR command?

It expects the data and you transfer it. Thereafter you check the
status and if it's ok all is well. Otherwise you get an error in
return. However such errors are very rare.

>After writing the command register and the other ones, the flash expects 
>512 bytes to be written (with A2 A1 A0 set to 0 0 0 , right?)

yes

>But, what happens if I write for example only 20 bytes, then I wait some 
>time by letting my cpu doing other things, and then write other 60 
>bytes, then I wait some time doing some calculations and finally write 
>the remaining 512-20-60 bytes?
>
>I mean... can I use the CF as a single sector buffer? Or just have I to 
>write all the 512B of a sector in a single shot with no interruptions?

I never had the need for such an aproach. The CF specs may state
somewhere a general timeout or such. Apart of this I don't see any
obvious reason why this should not work. I actually would expect a CF
card to act this way and as such this might make your live much
easier.

Markus

> I have a GPL ATA/ATAPI implementation via GPIO of Microchip PIC18. Its
> inner loop peaks at a bit under 2MB/sec (PIO) on a 40MHz (ext clock)
> PIC: http://www.telegraphics.com.au/svn/picide/trunk (Subversion).
> 

Is 2MB/sec the microcontroller's limit?  (or the peripheral limit?)
I'll have a look at the url, thank you very much.
Aldo

Markus Zingg wrote:
> this might make your live much
> easier.
> 

...hopefully :-)

Thanks once again, Markus! Bye,
Aldo

Haldo wrote:
> > I have a GPL ATA/ATAPI implementation via GPIO of Microchip PIC18. Its
> > inner loop peaks at a bit under 2MB/sec (PIO) on a 40MHz (ext clock)
> > PIC: http://www.telegraphics.com.au/svn/picide/trunk (Subversion).
> >
>
> Is 2MB/sec the microcontroller's limit?  (or the peripheral limit?)

That's as fast as the inner loop can possibly run, doing PIO with that
microcontroller (PIC18F6680). Happens to be a bit slower than the
slowest specified ATA Mode 0 timing.

--T

> I'll have a look at the url, thank you very much.
> Aldo

toby wrote:
> 
> That's as fast as the inner loop can possibly run, doing PIO with that
> microcontroller (PIC18F6680). Happens to be a bit slower than the
> slowest specified ATA Mode 0 timing.
> 

so, if the compact flash is slower, you have to wait for it to complete 
write cycles?

Haldo wrote:
> toby wrote:
> >
> > That's as fast as the inner loop can possibly run, doing PIO with that
> > microcontroller (PIC18F6680). Happens to be a bit slower than the
> > slowest specified ATA Mode 0 timing.
> >
>
> so, if the compact flash is slower, you have to wait for it to complete
> write cycles?

I was using hard drives and CD-ROM, but all necessary handshaking is in
the ATA standard.

According to CompactFlash Revision 3.0 the maximum transfer speed can be up 
to 66MB/sec and with a DMA transfer mode. This is great news for the future 
of Compactflash. With 16 parallel data pins and enough physical space to 
hold tera bytes of information, almost secures it to be the leader in 
high-end memory and OEM devices into the near future. Although data transfer 
speeds haven't reached this level yet, it is only because of the hardware 
design costs. Manufactures of the PCMCIA hardware interface need to update 
their hardware to support the current revision 3.0. Then, this will spawn 
the manufactures of memory cards to take advantage of these new speeds. A 
lot of people think that the read/write speed of Flash memory is a 
limitation of data transfer, however in theory, almost any speed can be 
achieved with different design layouts which include data buffers and 
massive parallel architectures.

It's not inconceivable that future revisions of the CompactFlash 
specifications will include data transfer speeds a magnitude better than the 
current standard.

Thomas


"Haldo" <haldo@somewhere.net> wrote in message 
news:d9os0g$h84$1@newsfeed.cineca.it...
> Hi all,
> in a previous thread we discussed about MMC timings and write speed.
> These memory do not seem to guarantee a minimum sustained write bandwidth.
>
> I was thinking of migrating to CompactFlash (more pins, but seem faster). 
> Producers claim a very high sustained write speed (>1MB/sec), but I'm not 
> so confident to obtain such values.
>
> Can someone, according to its experience, comment on the write performance 
> that can be obtained when a microcontroller controls a compact flash card 
> for data logging?
> (i.e. how much time do I have to wait for the flash to complete the sector 
> write after I send all the 512 bytes?)
>
> In my application I need to store about 120KB/sec, so if the whole 
> operations lasts some ms, I should be just fine!
>
>
> As usual any comment, suggestion or discussion is welcome!
>
> Thank you!Aldo