avoiding flash memory writes as much as possible

Hello-

Im not sure if this is the group to ask so forgive me if so
please point me to the right group.  Im working on an embedded
x86 board that has a 4G flash RAM and has a threshold of about 10k
writes.

Im writing software that, unless I have a better way to do it, writes
a file to the flash every 5 seconds, 24 hrs a day.  The file being
written
is then picked up by a second process and analyzed.  So Im trying
to avoid disk (flash memory) writes so I dont wear out the flash.

Ive been looking into named pipes or IPC msg sending but I think that
they too write to 'disk'.  My last resort is to use a client/server
approach
between the two processes, but Im wondering if that too doesnt,
at some level, write to 'disk' as well each time.  What is the best
approach
to this problem?

I would imagine you all have extensive experience in this area
and can let me know what is typically done in this situation.

Thanks

On Sat, 4 Apr 2009 09:51:54 -0700 (PDT), Bob <jeep@rahul.net> wrote:

>Hello-
>
>Im not sure if this is the group to ask so forgive me if so
>please point me to the right group.  Im working on an embedded
>x86 board that has a 4G flash RAM and has a threshold of about 10k
>writes.

10K writes to a given memory cell or 10K cumulative writes to the flash,
regardless of location? *Generally* the flash vendor implements a decent
leveling algorithm that spreads the "damage" out over the entire array.

>Im writing software that, unless I have a better way to do it, writes
>a file to the flash every 5 seconds, 24 hrs a day.  The file being
>written
>is then picked up by a second process and analyzed.  So Im trying
>to avoid disk (flash memory) writes so I dont wear out the flash.

Cache data to scratchpad RAM and do One Big Write to flash.

>Ive been looking into named pipes or IPC msg sending but I think that
>they too write to 'disk'.  My last resort is to use a client/server
>approach
>between the two processes, but Im wondering if that too doesnt,
>at some level, write to 'disk' as well each time.  What is the best
>approach
>to this problem?

There's rarely a single best approach, especially when significant
details (O/S, available RAM, types of RAM, ...) are missing.

In general (so this may not apply in a particular situation) use
short-life flash as read-only media, e.g., only to boot and to load
executables. Use "regular" RAM (probably SDRAM, given the presumed
architecture) as a write cache. Use replaceable flash (such as an SD
card or a USB stick) for data logging.

>I would imagine you all have extensive experience in this area
>and can let me know what is typically done in this situation.

Try not to get into this situation in the first place? Sounds like some
upstream design decisions weren't optimal.

-- 
Rich Webb     Norfolk, VA

Bob wrote:

> Hello-
> 
> Im not sure if this is the group to ask so forgive me if so
> please point me to the right group.  Im working on an embedded
> x86 board that has a 4G flash RAM and has a threshold of about 10k
> writes.
> 
> Im writing software that, unless I have a better way to do it, writes
> a file to the flash every 5 seconds, 24 hrs a day.  The file being
> written
> is then picked up by a second process and analyzed.  So Im trying
> to avoid disk (flash memory) writes so I dont wear out the flash.

Use RAMDISK. Or, better, use a sensible way of communication between the 
processes. There are many depending on the OS that you use.

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

h.
>
> Use RAMDISK. Or, better, use a sensible way of communication between the
> processes. There are many depending on the OS that you use.
>

As it turns out the Flash does do write leveling.  Im working for the
guy
who does the hardware, but this is his first embedded device also.  We
are using a Linux kernel 2.4.36 (because for now we are stuck at that
revision level for a while).

Ive only done desktop systems so Ill have to look into RAMDISK.

What form of communication would you suggest knowing its a linux
kernel?  Im very familiar with writing tcp/ip client/servers, but if
another
IPC is much smaller (code wise) Id have to figure it out since Ive
never
done an IPC.

On Sat, 4 Apr 2009 17:42:05 -0700 (PDT), Bob <jeep@rahul.net> wrote in
comp.arch.embedded:

> h.
> >
> > Use RAMDISK. Or, better, use a sensible way of communication between the
> > processes. There are many depending on the OS that you use.
> >
> 
> As it turns out the Flash does do write leveling.  Im working for the
> guy
> who does the hardware, but this is his first embedded device also.  We
> are using a Linux kernel 2.4.36 (because for now we are stuck at that
> revision level for a while).
> 
> Ive only done desktop systems so Ill have to look into RAMDISK.
> 
> What form of communication would you suggest knowing its a linux
> kernel?  Im very familiar with writing tcp/ip client/servers, but if
> another
> IPC is much smaller (code wise) Id have to figure it out since Ive
> never
> done an IPC.

My first suggestion would be to tell your hardware guy to find a flash
with a much larger write allowance.  100K to 300K is easily available
now.  Is this flash soldered to the board?

If you can put in a socket for SD cards, the flash can be easily
replaced when it wears out, and longer-lasting cards are appearing
constantly.  The same is true of compact flash cards, but they are
more expensive now and are going to stay that way.  SD has won the
flash card wars.

There are two other bits of advice I can offer for using Linux with
flash.

First to make sure you built the kernel so that it will run with a
swap partition, or make sure that you do set up a ram disk for the
swap partition to live on.

Second, for any file systems on flash, you want to mount them with
"noatime", and possibly "nodiratime".  Otherwise, any time you even
read a file from flash through the file system, the file system will
rewrite the directory entry for that file to update the last accessed
time.  Consult your system's documentation for the best way to do
this.

But 10K write cycles is certainly way too small for a flash used in an
embedded system, especially if it is soldered and not easily replaced.

-- 
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html

Bob schrieb:
> h.
>> Use RAMDISK. Or, better, use a sensible way of communication between the
>> processes. There are many depending on the OS that you use.
>>
> 
> As it turns out the Flash does do write leveling.  Im working for the
> guy
> who does the hardware, but this is his first embedded device also.  We
> are using a Linux kernel 2.4.36 (because for now we are stuck at that
> revision level for a while).

You may have a look at http://www.ramtron.com/.
These devices are like RAM - fast and with unlimited write cycles, but 
nonvolatile like eeproms.


-- 
Mit freundlichen Gr��en

Frank-Christian Kr�gel

On Apr 4, 9:32=A0pm, Jack Klein <jackkl...@spamcop.net> wrote:
> On Sat, 4 Apr 2009 17:42:05 -0700 (PDT), Bob <j...@rahul.net> wrote in
> comp.arch.embedded:

> > h.
>
> > > Use RAMDISK. Or, better, use a sensible way of communication between =
the
> > > processes. There are many depending on the OS that you use.
>
> > As it turns out the Flash does do write leveling. =A0Im working for the
> > guy
> > who does the hardware, but this is his first embedded device also. =A0W=
e
> > are using a Linux kernel 2.4.36 (because for now we are stuck at that
> > revision level for a while).
>
> > Ive only done desktop systems so Ill have to look into RAMDISK.

Be sure to build your root file system on ramdisk.  Otherwise, you
will write to flash periodically even if you don't do anything.

>
> > What form of communication would you suggest knowing its a linux
> > kernel? =A0Im very familiar with writing tcp/ip client/servers, but if
> > another
> > IPC is much smaller (code wise) Id have to figure it out since Ive
> > never
> > done an IPC.

You take care of the root file system and it will take care of system
buffers and IPCs.

>
> My first suggestion would be to tell your hardware guy to find a flash
> with a much larger write allowance. =A0100K to 300K is easily available
> now. =A0Is this flash soldered to the board?

Yes, even old CF can write 100K cycles or more.

>
> If you can put in a socket for SD cards, the flash can be easily
> replaced when it wears out, and longer-lasting cards are appearing
> constantly. =A0The same is true of compact flash cards, but they are
> more expensive now and are going to stay that way. =A0SD has won the
> flash card wars.

But small CFs are much cheaper and easier to run as IDE.  We run small
disposal CFs as primary file system and larger read only CF as data
storage.  We replace the $1 CF every few months, just in case.  The
larger 1G,2G,4G CF lasts for years.

On Sat, 04 Apr 2009 09:51:54 -0700, Bob wrote:

> Im not sure if this is the group to ask so forgive me if so
> please point me to the right group.  Im working on an embedded
> x86 board that has a 4G flash RAM and has a threshold of about 10k
> writes.
> 
> Im writing software that, unless I have a better way to do it, writes
> a file to the flash every 5 seconds, 24 hrs a day.  The file being written
> is then picked up by a second process and analyzed.  So Im trying
> to avoid disk (flash memory) writes so I dont wear out the flash.
> 
> Ive been looking into named pipes or IPC msg sending but I think that
> they too write to 'disk'.

They don't.

> My last resort is to use a client/server approach
> between the two processes, but Im wondering if that too doesnt,
> at some level, write to 'disk' as well each time.  What is the best approach
> to this problem?

This would normally be done using either shared memory or Unix-domain
sockets.

But if you want to retain your existing design, you can just create a RAM
disk and store the file there.

Hi Bob.

> Ive only done desktop systems so Ill have to look into RAMDISK.
>
> What form of communication would you suggest knowing its a linux
> kernel? =A0Im very familiar with writing tcp/ip client/servers, but if
> another
> IPC is much smaller (code wise) Id have to figure it out since Ive
> never done an IPC.

If you want to leave it as a file exchange:
         mount -t tmpfs -o size=3D10M none /mnt

You may already find that /tmp is created as a tmpfs file system.

             Steve Schefter