Guess how many bad blocks are typical for NAND flash of several GB 
capacity? As many as 2 percent! There could be the whole areas of 
hundreds of megabytes of the contiguous bad cells, as well as the random 
scatter.

It is possible to do the extensive read/write test to find the most of 
the unreliable blocks; but it takes many hours.

I didn't encounter this problem until we started to use the high 
capacity CF cards. The bad blocks were very rare for the cards of 1GB 
and below. Since the flash iself is hidden behind the IDE interface and 
a compatible file system, and the read/write performance is critical, it 
is generally impossible to apply an error correction scheme.

I was under impression that flash is more reliable then HDD; now I see 
that it is not so. Do you know how reliable are the IDE flash drives?


Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

Vladimir Vassilevsky wrote:
> ...
>
> I was under impression that flash is more reliable then HDD; now I see
> that it is not so. Do you know how reliable are the IDE flash drives?
>

Vladimir,
if flash were a viable and reliable replacement for HDDs this would
have
happened for years by now, the costs would have gone down. They are
not, and given their limited number of write cycles they are bound to
stay out of the way of normal disks (which have achieved an amazing
level of performance).
 Thanks for posting the details you measured, I'll know now to not buy
a 2G SD-card for my camera (not that I need any more than the 1G I
have now, I have never used > 1/3 its capacity anyway :-) ).

Didi

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

Original message: http://groups.google.com/group/comp.arch.embedded/msg/e9e96546454a52ab?dmode=source

On Jun 27, 12:10 pm, Vladimir Vassilevsky <antispam_bo...@hotmail.com>
wrote:
> Guess how many bad blocks are typical for NAND flash of several GB
> capacity? As many as 2 percent! There could be the whole areas of
> hundreds of megabytes of the contiguous bad cells, as well as the random
> scatter.

I thought the IDE interface was supposed to hide that from the host by
mapping in spares?

Isn't it also supposed to do wear leveling behind your back?

On Jun 27, 12:29 pm, Didi <d...@tgi-sci.com> wrote:

> Vladimir,
> if flash were a viable and reliable replacement for HDDs this would
> have
> happened for years by now, the costs would have gone down. They are
> not, and given their limited number of write cycles they are bound to
> stay out of the way of normal disks (which have achieved an amazing
> level of performance).

Flash is at the moment becoming a viable replacement for many
applications, both  high end and low end - witness flash disks turning
up in everything from servers, to high end ultraportable notebooks, to
low end ones like the EeePC and even cheaper competitors.

This wasn't reasonable until the most recent generations of devices
started beating the performance and price point of the 1.8" mechanical
drives.

I don't necessarily think flash will be a replacement for large, cheap
mechanical drives, but for applications that only need a few GB, or
for applications where space and weight count and 32-64 GB is enough,
it's already gaining market penetration.

Write cycles could be an issue, but many of the ultraportable gadgetry
applications will see system replacement before that happens.  And the
replacement will probably be 2 - 4 x the GB/$ of the original.

c...@hotmail.com wrote:


>>Guess how many bad blocks are typical for NAND flash of several GB
>>capacity? As many as 2 percent! There could be the whole areas of
>>hundreds of megabytes of the contiguous bad cells, as well as the random
>>scatter.
> 
> I thought the IDE interface was supposed to hide that from the host by
> mapping in spares?

It clearly does that, but only to a limited amount. Also, when a file 
gets broken because of the bad block, it is too late to replace it.

> Isn't it also supposed to do wear leveling behind your back?

There is a whole lot of things that the internal controller could 
probably do; but we can only guess about what it actually does.


Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

Vladimir Vassilevsky <a...@hotmail.com> writes:

> c...@hotmail.com wrote:
>
>
>>>Guess how many bad blocks are typical for NAND flash of several GB
>>>capacity? As many as 2 percent! There could be the whole areas of
>>>hundreds of megabytes of the contiguous bad cells, as well as the random
>>>scatter.
>>
>> I thought the IDE interface was supposed to hide that from the host by
>> mapping in spares?
>
> It clearly does that, but only to a limited amount. Also, when a file
> gets broken because of the bad block, it is too late to replace it.

Just because the device is manufactured with bad blocks, does not
*automatically* mean that more will follow. (At least you have not
demonstrated that).

It's not like a load of bad blocks appearing on a magnetic drive,
where you know the whole drive is probably on the way out.

>> Isn't it also supposed to do wear leveling behind your back?
>
> There is a whole lot of things that the internal controller could
> probably do; but we can only guess about what it actually does.

It's the same for conventional disks isn't it?

-- 

John Devereux

Didi wrote:
> Vladimir Vassilevsky wrote:
>> ...
>>
>> I was under impression that flash is more reliable then HDD; now I see
>> that it is not so. Do you know how reliable are the IDE flash drives?
>>
> 
> Vladimir,
> if flash were a viable and reliable replacement for HDDs this would
> have
> happened for years by now, the costs would have gone down. They are
> not, and given their limited number of write cycles they are bound to
> stay out of the way of normal disks (which have achieved an amazing
> level of performance).
>  Thanks for posting the details you measured, I'll know now to not buy
> a 2G SD-card for my camera (not that I need any more than the 1G I
> have now, I have never used > 1/3 its capacity anyway :-) ).

It also explains why the 8GB Photo Hard
Drive I bought for my Nikon is rock-solid
and I've had many intermittent and non-
reproduceable problems with larger FLASH
cards.

On Jun 27, 11:10=A0am, Vladimir Vassilevsky <antispam_bo...@hotmail.com>
wrote:
> Guess how many bad blocks are typical for NAND flash of several GB
> capacity? As many as 2 percent! There could be the whole areas of
> hundreds of megabytes of the contiguous bad cells, as well as the random
> scatter.


FWIW, for an 4GB flash device, 2% would be 80MB.

You'll also notice that most (all?) flash manufactures follow the
"1GB=3D1,000,000,000 bytes" rule common for disk drives.  Leaves them
plenty of space (6.8%, in fact) for manufacturing defects and wear
leveling.

Didi schreef:
> Vladimir Vassilevsky wrote:
>> ...
>>
>> I was under impression that flash is more reliable then HDD; now I see
>> that it is not so. Do you know how reliable are the IDE flash drives?
>>
> 
> Vladimir,
> if flash were a viable and reliable replacement for HDDs this would
> have
> happened for years by now, the costs would have gone down. They are
> not, and given their limited number of write cycles they are bound to
> stay out of the way of normal disks (which have achieved an amazing
> level of performance).

When the platter densities of (mechanical) HDD's went up at a certain 
point manufactures of HDD's had to resort to error correction schemes to 
obtain reliable operation. A modern HDD would be unusable without ECC. 
It appears that high density flash is going the same direction.

Vladimir Vassilevsky wrote:
> 
> Guess how many bad blocks are typical for NAND flash of several GB 
> capacity? As many as 2 percent! There could be the whole areas of 
> hundreds of megabytes of the contiguous bad cells, as well as the random 
> scatter.
> 
> It is possible to do the extensive read/write test to find the most of 
> the unreliable blocks; but it takes many hours.
> 
> I didn't encounter this problem until we started to use the high 
> capacity CF cards. The bad blocks were very rare for the cards of 1GB 
> and below. Since the flash iself is hidden behind the IDE interface and 
> a compatible file system, and the read/write performance is critical, it 
> is generally impossible to apply an error correction scheme.
> 
> I was under impression that flash is more reliable then HDD; now I see 
> that it is not so. Do you know how reliable are the IDE flash drives?
> 

NAND flash always has defects in manufacturing - the devices are 
designed to cope with a certain level of faults to make manufacturing 
cheaper (the same applies to many other types of chips, and hard disks). 
  Each sector in NAND has extra space for error correction and detection 
(IIRC, 512 byte sectors are actually 528 bytes in size).  Bad blocks can 
be detected and marked during manufacture and testing, and blocks that 
go bad (due to wearing out) are detected in use and the data moved to 
different blocks.

The same thing is done with hard disks - the controller detects bad 
blocks, and re-maps them.  There are a few differences, however - on 
hard disks, you get bad blocks in manufacturing but it is rare that a 
good block goes bad in use.  With flash, the controller can almost 
always spot a bad block and recover the data (since it's normally a 
single bit failure, the ECC will fix it), while on a hard disk you lose 
data.  And on flash, a remapping makes no difference to performance - on 
a hard disk, it's equivalent to file fragmentation.

CF cards and other earlier flash devices are not that great at wear 
levelling and bad block handling (that's one of the reasons for 
flash-specific file systems like YAFS and JFFS2).  Modern IDE, SATA and 
SAS flash drives are far better.  Good manufacturers quote MTBF numbers 
that are orders of magnitude higher than for hard disks, and wear is no 
longer a practical issue for larger flash disks (I've seen flash disks 
spec'ed for *continuous* 20 MB/s writes for years).  See also 
<http://wiki.eeeuser.com/ssd_write_limit>; - a 4 GB Eee PC disk should be 
fine for a normal user for 25 years.  And since wear is levelled across 
a disk, a 128 GB disk will survive 32 times as much use for the same time.