Serial EEPROM or Serial Flash?

I need to save some data on a non-volatile memory. They are some 
parameters that the user could change infrequently, for example 10 times 
per day at a maximum.  In the range 100-2kB.

As usually occurs, the parameters that changes more frequently (10 times 
per day) are fewer than parameters that changes very rarely (10-100 
times in the device lifetime).

How to save those data? After discarding the internal MCU Flash (because 
of interrupts block during programming), I'm deciding if it's better a 
serial EEPROM or serial Flash.

First of all, I think SPI is better than I2C.  SPI seems much faster: 
10-20MHz against 400kHz-1MHz.  At least for reading.  Erasing/writing 
time is identical between I2C and SPI.

EEPROM or Flash? I know EEPROM can be written one byte at a time without 
erasing an entire block, against Flash that needs a sector erase before 
writing even a single byte.

The firmware would be simpler with EEPROMs, because I don't need to save 
the entire sector before erasing and restoring it during programming, 
when writing a single byte.  With EEPROMs I can write a byte.  Stop.

However I don't think this simple approach can be used in a real 
production.  Suppose I have 10 bytes to write.  What happens if the 
writing process is stopped at the middle, maybe after 5 bytes?  How to 
protect the system against those events?  I think one solution is to 
have at least two copies of data in the memory and switch to the other 
bank after all data is completely written, with an "atomic" write operation.
This means I need to copy&paste an entire block everytime, even for a 
single byte change.  And this is similar to Flash approach, where I 
*need* a sector erase before changing a single byte.

What about the time?  EEPROM write cycle is about 5ms for a 32-bytes 
page. For 128 bytes/4 pages, 20 ms.
Flash sector erase time is 18 ms, plus 14us for each byte. The overall 
write cycle time is similar between EEPROM and Flash.

If the data are bigger, for example 1kB, the Flash technology wins.  The 
sector size in Flash memories are usually bigger than 1kB.  So I need to 
erase only one time (18m + 14u*1024=32ms).  In EEPROM I have 32 32-bytes 
pages, so 5m*32=160ms.  5 times more than Flash.

I'm not considering endurance.  EEPROMs are better (1000k write cycles) 
than Flash (100k write cycles), but I don't need so much write cycles in 
the entire device lifetime.


After those considerations, I think I will choose a serial SPI Flash, 
maybe 512kbit.  The price is similar to serial EEPROM of only 64kbit.
I see only one difficulty: how to write a good device driver to manage 
serial Flash: copy the content of a sector in another sector, making the 
changes on-the-fly, and switch from one sector to the other in one 
"atomic" operation.  Could you suggest some code to study?

I missed something other?  Please share your considerations.

On 14/06/18 12:20, pozz wrote:
> I need to save some data on a non-volatile memory. They are some
> parameters that the user could change infrequently, for example 10 times
> per day at a maximum.  In the range 100-2kB.
> 
> As usually occurs, the parameters that changes more frequently (10 times
> per day) are fewer than parameters that changes very rarely (10-100
> times in the device lifetime).
> 
> How to save those data? After discarding the internal MCU Flash (because
> of interrupts block during programming), I'm deciding if it's better a
> serial EEPROM or serial Flash.
> 
> First of all, I think SPI is better than I2C.  SPI seems much faster:
> 10-20MHz against 400kHz-1MHz.  At least for reading.  Erasing/writing
> time is identical between I2C and SPI.

Read time for this sort of thing is usually irrelevant.  Assuming you
are not badly constrained for ram, you read the NVM structure into
memory at startup and use it from there.  This makes it far easier to
use, and to handle things like duplicate structures in the NVM with
timestamps, checksums, etc.

> 
> EEPROM or Flash? I know EEPROM can be written one byte at a time without
> erasing an entire block, against Flash that needs a sector erase before
> writing even a single byte.

Roughly correct.  It is not an exact difference - there are devices
marketed as E�PROM which are written in, say, 32-byte blocks.  And there
are "flash" devices where you can write a single byte, and the device
handles reading the old page into a buffer, updating the byte, erasing
the page and re-writing.

> 
> The firmware would be simpler with EEPROMs, because I don't need to save
> the entire sector before erasing and restoring it during programming,
> when writing a single byte.  With EEPROMs I can write a byte.  Stop.
> 
> However I don't think this simple approach can be used in a real
> production.  Suppose I have 10 bytes to write.  What happens if the
> writing process is stopped at the middle, maybe after 5 bytes?  How to
> protect the system against those events?  I think one solution is to
> have at least two copies of data in the memory and switch to the other
> bank after all data is completely written, with an "atomic" write
> operation.

Multiple copies (at least 2) are key, along with timestamps or counters
and checksums.

> This means I need to copy&paste an entire block everytime, even for a
> single byte change.  And this is similar to Flash approach, where I
> *need* a sector erase before changing a single byte.

Yes.

Of course, it is possible to have multiple smaller blocks rather than
just one big one.  And you don't have to erase the old blocks until you
need to re-claim the space.

> 
> What about the time?  EEPROM write cycle is about 5ms for a 32-bytes
> page. For 128 bytes/4 pages, 20 ms.
> Flash sector erase time is 18 ms, plus 14us for each byte. The overall
> write cycle time is similar between EEPROM and Flash.
> 
> If the data are bigger, for example 1kB, the Flash technology wins.  The
> sector size in Flash memories are usually bigger than 1kB.  So I need to
> erase only one time (18m + 14u*1024=32ms).  In EEPROM I have 32 32-bytes
> pages, so 5m*32=160ms.  5 times more than Flash.
> 
> I'm not considering endurance.  EEPROMs are better (1000k write cycles)
> than Flash (100k write cycles), but I don't need so much write cycles in
> the entire device lifetime.
> 
> 
> After those considerations, I think I will choose a serial SPI Flash,
> maybe 512kbit.  The price is similar to serial EEPROM of only 64kbit.
> I see only one difficulty: how to write a good device driver to manage
> serial Flash: copy the content of a sector in another sector, making the
> changes on-the-fly, and switch from one sector to the other in one
> "atomic" operation.  Could you suggest some code to study?
> 
> I missed something other?  Please share your considerations.

Il 14/06/2018 12:49, David Brown ha scritto:
> On 14/06/18 12:20, pozz wrote:
>> I need to save some data on a non-volatile memory. They are some
>> parameters that the user could change infrequently, for example 10 times
>> per day at a maximum.  In the range 100-2kB.
>>
>> As usually occurs, the parameters that changes more frequently (10 times
>> per day) are fewer than parameters that changes very rarely (10-100
>> times in the device lifetime).
>>
>> How to save those data? After discarding the internal MCU Flash (because
>> of interrupts block during programming), I'm deciding if it's better a
>> serial EEPROM or serial Flash.
>>
>> First of all, I think SPI is better than I2C.  SPI seems much faster:
>> 10-20MHz against 400kHz-1MHz.  At least for reading.  Erasing/writing
>> time is identical between I2C and SPI.
> 
> Read time for this sort of thing is usually irrelevant.  Assuming you
> are not badly constrained for ram, you read the NVM structure into
> memory at startup and use it from there.  

In my actual project I have 2kB of non volatile data and a MCU with 32kB 
of RAM.  I think I can load all in RAM at startup... even if it's silly, 
because there's an array of 100 users that I don't usually need.  I only 
need to search for a valid user when one login attempt happens.


> This makes it far easier to
> use, and to handle things like duplicate structures in the NVM with
> timestamps, checksums, etc.

Why?  I think it is easier because, if you have all data in RAM, you 
need to only write during normal execution (the reading is performed 
only at startup).  In this case, you don't mix reading/writing during 
execution.


>> EEPROM or Flash? I know EEPROM can be written one byte at a time without
>> erasing an entire block, against Flash that needs a sector erase before
>> writing even a single byte.
> 
> Roughly correct.  It is not an exact difference - there are devices
> marketed as E�PROM which are written in, say, 32-byte blocks.  

Anyway you can change only one byte of a 32-byte pages.


> And there
> are "flash" devices where you can write a single byte, and the device
> handles reading the old page into a buffer, updating the byte, erasing
> the page and re-writing.

Hmm... strange chip.


>> The firmware would be simpler with EEPROMs, because I don't need to save
>> the entire sector before erasing and restoring it during programming,
>> when writing a single byte.  With EEPROMs I can write a byte.  Stop.
>>
>> However I don't think this simple approach can be used in a real
>> production.  Suppose I have 10 bytes to write.  What happens if the
>> writing process is stopped at the middle, maybe after 5 bytes?  How to
>> protect the system against those events?  I think one solution is to
>> have at least two copies of data in the memory and switch to the other
>> bank after all data is completely written, with an "atomic" write
>> operation.
> 
> Multiple copies (at least 2) are key, along with timestamps or counters

Two should be sufficient to prevent corruption caused by interruption 
during the writing of a block of data.
Maybe three (or more) are needed to face memory *physical* corruption 
(maybe for many writings).
I think I can ignore this event in my actual project.


> and checksums.

This is interesting.  Why do I need a checksum?  My approach is to use 
only a magic number plus a counter... and two memory areas.
At first startup magic number isn't found on any areas, so the device 
starts with default and write data on Area1 (magic/0, where 0 is the 
counter).

When the configuration is changed, Area2 is written with magic/1, being 
careful to save magic/1 only at the end of area writing.

At startup magics and counters from both area are loaded, and one area 
is chosen (magic should be valid and counter should be the maximum).

I think this approach works, even when the area writing is interrupted 
at the middle.

Why do I need checksum?  The only thing that comes in mind is to prevent 
writing errors: for example, I want to write 0x00 but the value really 
written is 0x01, maybe for a noise on the serial bus.

To solve this situation, I need checksum... but also I need to re-read 
and re-calculate the checksum at *every* area writing... and start a new 
writing if something was wrong.

Do you have a better strategy?


>> This means I need to copy&paste an entire block everytime, even for a
>> single byte change.  And this is similar to Flash approach, where I
>> *need* a sector erase before changing a single byte.
> 
> Yes.
> 
> Of course, it is possible to have multiple smaller blocks rather than
> just one big one.  

This is good to reduce writing time of a block of data.
In my case I have 2kB of data, around 62 32-bytes EEPROM pages, that is 
5*62=300ms.  I don't like to block a task (in a cooperative OS) for so 
long time.

So I could thing to split data in smaller blocks.  Maybe I can accept a 
task that blocks for 50ms maximum.  This means 10 pages, 320 bytes.  So 
my data should be splitted in 6 areas.

I think they are too much.


> And you don't have to erase the old blocks until you
> need to re-claim the space.

Surely... however at steady-state I have only messed areas that should 
be erased at every writing.


>> What about the time?  EEPROM write cycle is about 5ms for a 32-bytes
>> page. For 128 bytes/4 pages, 20 ms.
>> Flash sector erase time is 18 ms, plus 14us for each byte. The overall
>> write cycle time is similar between EEPROM and Flash.
>>
>> If the data are bigger, for example 1kB, the Flash technology wins.  The
>> sector size in Flash memories are usually bigger than 1kB.  So I need to
>> erase only one time (18m + 14u*1024=32ms).  In EEPROM I have 32 32-bytes
>> pages, so 5m*32=160ms.  5 times more than Flash.
>>
>> I'm not considering endurance.  EEPROMs are better (1000k write cycles)
>> than Flash (100k write cycles), but I don't need so much write cycles in
>> the entire device lifetime.
>>
>>
>> After those considerations, I think I will choose a serial SPI Flash,
>> maybe 512kbit.  The price is similar to serial EEPROM of only 64kbit.
>> I see only one difficulty: how to write a good device driver to manage
>> serial Flash: copy the content of a sector in another sector, making the
>> changes on-the-fly, and switch from one sector to the other in one
>> "atomic" operation.  Could you suggest some code to study?
>>
>> I missed something other?  Please share your considerations.
> 

Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:

> Why do I need checksum?  The only thing that comes in mind is to prevent 
> writing errors: for example, I want to write 0x00 but the value really 
> written is 0x01, maybe for a noise on the serial bus.
> 
> To solve this situation, I need checksum... but also I need to re-read 
> and re-calculate the checksum at *every* area writing... and start a new 
> writing if something was wrong.

At power up, when you read Area1 (for example) how do you know that the data are not corrupted (even if magic/counter are correct)?

Bye Jack

AT Friday 15 June 2018 15:54, Jack wrote:

> Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
> 
>> Why do I need checksum?  The only thing that comes in mind is to prevent
>> writing errors: for example, I want to write 0x00 but the value really
>> written is 0x01, maybe for a noise on the serial bus.
>> 
>> To solve this situation, I need checksum... but also I need to re-read
>> and re-calculate the checksum at *every* area writing... and start a new
>> writing if something was wrong.
> 
> At power up, when you read Area1 (for example) how do you know that the
> data are not corrupted (even if magic/counter are correct)?
> 
> Bye Jack

How can he know if the write was successful and not aborted midway because 
of a power interruption.
With those flash devices who have their "own life" you don’t even have a 
guaranty that things get written in the same sequence as the software 
thinks. So writing the "magic" after the data to make sure it signals 
updated data does not guaranty that physically it gets written before the 
data.

-- 
Reinhardt

Il giorno venerdì 15 giugno 2018 10:26:42 UTC+2, Reinhardt Behm ha scritto:
> AT Friday 15 June 2018 15:54, Jack wrote:
> 
> > Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
> > 
> >> Why do I need checksum?  The only thing that comes in mind is to prevent
> >> writing errors: for example, I want to write 0x00 but the value really
> >> written is 0x01, maybe for a noise on the serial bus.
> >> 
> >> To solve this situation, I need checksum... but also I need to re-read
> >> and re-calculate the checksum at *every* area writing... and start a new
> >> writing if something was wrong.
> > 
> > At power up, when you read Area1 (for example) how do you know that the
> > data are not corrupted (even if magic/counter are correct)?
> > 
> > Bye Jack
> 
> How can he know if the write was successful and not aborted midway because 
> of a power interruption.
> With those flash devices who have their "own life" you don’t even have a 
> guaranty that things get written in the same sequence as the software 
> thinks. So writing the "magic" after the data to make sure it signals 
> updated data does not guaranty that physically it gets written before the 
> data.

I suppose the timing of the write are known (and/or there is a pin/status register that will tell if the write has ended), so just wait a little bit and read back the data.

Bye Jack

Il 15/06/2018 09:54, Jack ha scritto:
> Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
> 
>> Why do I need checksum?  The only thing that comes in mind is to prevent
>> writing errors: for example, I want to write 0x00 but the value really
>> written is 0x01, maybe for a noise on the serial bus.
>>
>> To solve this situation, I need checksum... but also I need to re-read
>> and re-calculate the checksum at *every* area writing... and start a new
>> writing if something was wrong.
> 
> At power up, when you read Area1 (for example) how do you know that the data are not corrupted (even if magic/counter are correct)?

In my application I think I can ignore two problematic events:

- the value really written is different for a noise on the bus signals;
- the value really written is different for physical damage of the memory.

If I ignore these events/situations, I think checksums aren't needed and 
I can consider only magic/counter.

Il 15/06/2018 10:26, Reinhardt Behm ha scritto:
> AT Friday 15 June 2018 15:54, Jack wrote:
> 
>> Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
>>
>>> Why do I need checksum?  The only thing that comes in mind is to prevent
>>> writing errors: for example, I want to write 0x00 but the value really
>>> written is 0x01, maybe for a noise on the serial bus.
>>>
>>> To solve this situation, I need checksum... but also I need to re-read
>>> and re-calculate the checksum at *every* area writing... and start a new
>>> writing if something was wrong.
>>
>> At power up, when you read Area1 (for example) how do you know that the
>> data are not corrupted (even if magic/counter are correct)?
>>
>> Bye Jack
> 
> How can he know if the write was successful and not aborted midway because
> of a power interruption.

Because of the counter that is written as the last byte.  If the write 
has interrupted, the counter of the new area contains an old value, so 
at startup is loaded old area.

> With those flash devices who have their "own life" you don’t even have a
> guaranty that things get written in the same sequence as the software
> thinks. So writing the "magic" after the data to make sure it signals
> updated data does not guaranty that physically it gets written before the
> data.

I can't follow your arguments... if I send write commands in a sequence, 
how the data could be written with a different sequence?

AT Friday 15 June 2018 16:50, pozz wrote:

> Il 15/06/2018 10:26, Reinhardt Behm ha scritto:
>> AT Friday 15 June 2018 15:54, Jack wrote:
>> 
>>> Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
>>>
>>>> Why do I need checksum?  The only thing that comes in mind is to
>>>> prevent writing errors: for example, I want to write 0x00 but the value
>>>> really written is 0x01, maybe for a noise on the serial bus.
>>>>
>>>> To solve this situation, I need checksum... but also I need to re-read
>>>> and re-calculate the checksum at *every* area writing... and start a
>>>> new writing if something was wrong.
>>>
>>> At power up, when you read Area1 (for example) how do you know that the
>>> data are not corrupted (even if magic/counter are correct)?
>>>
>>> Bye Jack
>> 
>> How can he know if the write was successful and not aborted midway
>> because of a power interruption.
> 
> Because of the counter that is written as the last byte.  If the write
> has interrupted, the counter of the new area contains an old value, so
> at startup is loaded old area.
> 
>> With those flash devices who have their "own life" you don’t even have a
>> guaranty that things get written in the same sequence as the software
>> thinks. So writing the "magic" after the data to make sure it signals
>> updated data does not guaranty that physically it gets written before the
>> data.
> 
> I can't follow your arguments... if I send write commands in a sequence,
> how the data could be written with a different sequence?

If this is a flash device with an internal controller that does the 
blocking/unblocking for you - meaning to change a single byte it reads a 
block, changes the byte, erases the block and writes the block - it has its 
own life. You must know that it does not cache things and "optimizes" writes 
by reordering them. Some flash drives are just doing this.
If it is a dump device then you are on the save side. But you must make 
sure.
With the usual EEPROMs (like the small 8-pin I2C devices) you are on the 
save side. They are dump. But I would never trust these flash chips without 
having a clear specification from the manufacturer about how they work. Some 
manufacturers do strange things to get nice "performance" numbers for 
marketing.

-- 
Reinhardt

Il giorno venerdì 15 giugno 2018 10:47:55 UTC+2, pozz ha scritto:
> Il 15/06/2018 09:54, Jack ha scritto:
> > Il giorno venerdì 15 giugno 2018 09:39:03 UTC+2, pozz ha scritto:
> > 
> >> Why do I need checksum?  The only thing that comes in mind is to prevent
> >> writing errors: for example, I want to write 0x00 but the value really
> >> written is 0x01, maybe for a noise on the serial bus.
> >>
> >> To solve this situation, I need checksum... but also I need to re-read
> >> and re-calculate the checksum at *every* area writing... and start a new
> >> writing if something was wrong.
> > 
> > At power up, when you read Area1 (for example) how do you know that the data are not corrupted (even if magic/counter are correct)?
> 
> In my application I think I can ignore two problematic events:
> 
> - the value really written is different for a noise on the bus signals;
> - the value really written is different for physical damage of the memory.

You can ignore these events only if you don't care about the data.
Else you need to find a strategy manage the errors (in general this is easy: just read back the data after you're sure that the write ended).

> If I ignore these events/situations, I think checksums aren't needed and 
> I can consider only magic/counter.

The issue is not after write, the issue is after powerup, when you read the data from flash, you don't know if the data is correct or corrupted, even if the magic number/counter a consistent.
A checksum is a way to be sure of this (but you can't recover if something is corrupted). Another is to write the same data multiple times and then use a mojority vote to decide if they are correct or not (and recover if there is something corrupted).

Bye Jack