Hello,

I am implementing data logging on an SD card using the SPI interface and FATFS libraries, with reference to the SD card libraries provided by Controllers Tech (https://controllerstech.com/sd-card-using-spi-in-s...). While I have achieved a logging speed of around 58 Hz, I am encountering data loss issues, particularly at higher logging speeds. I would appreciate any suggestions or improvements to enhance the reliability of the implementation on the microcontroller.

Sd card function:

void HandleState(void) {

    uint8_t path[20];

    switch (currentState) {

        case STATE_INIT:

            if (f_mount(&fs, "0:/", 1) == FR_OK) {

                int fileNumber = 0;

                do {

                    sprintf((char *)path, "log_%d.csv", fileNumber++);

                } while (f_stat((char *)path, NULL) == FR_OK);

                if (f_open(&file, (char *)path, FA_CREATE_ALWAYS | FA_WRITE | FA_READ) == FR_OK) {

                    strcpy(buffer, "Log SD card\n");

                    f_write(&file, buffer, sizeof(buffer), NULL);

                    currentState = STATE_LOG_DATA;

                } else {

                    Error_Handler();

                }

            } else {

                Error_Handler();

            }

            break;

        case STATE_LOG_DATA:

                sprintf(buffer, "%lu, %.2f, %d\n", HAL_GetTick(), temperature, counter++);  // current time in milliseconds since the start of the program

                HAL_GPIO_WritePin(GPIOC, GPIO_PIN_0, SET);

                f_write(&file, buffer, sizeof(buffer), NULL);

                f_sync(&file) ;

                HAL_GPIO_WritePin(GPIOC, GPIO_PIN_0, RESET);

            break;

        default:

            break;

    }

}

My _guess_ is that whenever the SD card routines need to erase a sector of the flash memory that there is a very long delay (perhaps hundreds of milliseconds) before you can access the SD card again.  If you don't want to lose information during that time, you need to store logging information in a RAM buffer during that time and write the logging routines so they can read from that buffer and catch up again afterward.  But as always, you should measure before you optimize.  So...

The first step might be to put an oscilloscope (or cheap USB logic analyzer) on GPIO_PIN_0.  How long is it set for each time?  How long does it spend doing other stuff before it gets set again?  Is the timing nice and consistent, or are there occasional long pulses or gaps?

If you don't have a scope, you can see how long things take by storing the value read from a high-resolution timer (your "counter" might be good enough) at the beginning and end of your logging function.  Subtract the beginning time from the end time to see how long that function takes and store the result in a circular buffer.  Make the stored time variables static or global and subtract the logging function's end time from the _next_ start time to see how long it is before the logging function gets called again (and store that in a different circular buffer).  Occasionally print out the contents of those buffers or pause the program and view them with a debugger.  Note that just storing the values rather than printing/logging them as you go is relatively fast and keeps the added measurement code from affecting the timing of your program too much.

You could try omitting the call to f_sync() to see what effect that has.

You could write the data in a more compact binary form so it takes less time to format and write.  Of course, you then need a way to decode and display it.  But you can fit more on the SD card.

If there are things that are fast enough on average but occasionally take a long time to finish and cause you to miss events (logging or otherwise), you could consider doing them in another thread (if you have an RTOS) or using a state machine that does part of the job each time it is called but quickly exits when it can't make progress because, for example, it needs to wait for erasing a sector of flash memory to finish.  That might require digging into the low-level routines behind f_write() and f_sync(), adding buffers, and making your code or an interrupt keep updating a state machine that reads from the buffer and writes to the SD card.

Steve

I agree with skstrobel. Use the GPIO to check timing. In your linked page in the comments section Antonio writes:

"I managed to get it working and I measured the time it takes for the writing instructions. In my case is around 3-4 us, but after every 128 writing instructions, it takes 5ms to perform the next write.

Is this the expected behavior? or is there a workaround to fix this?"

And it appears other users have the same issue. The cause of this delay isn't noted though.

Like skstrobel says, removing f_sync() might allow the driver to buffer data during those periods. Typically f_sync() would be found in a STATE_LOG_CLOSE state along with f_close(). Depending on f_sync() to perform a sort-of running f_close() is unlikely to work consistently.

I'm not familiar with this particular driver, but typically a write() function returns the number of bytes written or -1 on error. If that's the case here, it's also good practice to verify that the 'number of bytes written' that f_write() returns equals 'sizeof(buffer)' and if not to retry the unwritten bytes, as well as to handle any errors if -1 is returned.

"I managed to get it working and I measured the time it takes for the writing instructions. In my case is around 3-4 us, but after every 128 writing instructions, it takes 5ms to perform the next write.

Is this the expected behavior? or is there a workaround to fix this?"

That sounds about right. NAND FLASH (what SD cards use) can write relative small units, but can erase only pretty large blocks. For SD cards the read/write unit (sector) size is 512 bytes (at least since v2.0) and a common erase block size is 64K, i.e. 128 sectors. Since with FLASH you can only turn a '1' bit to '0' during write and a '0' to '1' during erase, you can only write to a previously erased area. So as you writing sector after sector, when a previously erased block gets full, the card needs to erase the next 64K block before it can write the next sector to it. After that you have 128 erased sectors, which you can write pretty fast, then the process repeats. There are all sorts of erase commands so that you can pre-emptively erase large areas before you write them, but you still need buffering, if your source cannot wait (e.g. recording sound or video).

So it's not the fatfs or the SD driver, it's the way SD cards work.

1) Can you show the whole code? How many times your function is called?

2) How big is your buffer? With:

f_write(&file, buffer, sizeof(buffer), NULL);

you write the whole buffer. Better use strlen(buffer).

3) Why you open in read and write mode? Write mode is enough here.

Eric