UTF-8 is a variable-width character encoding that represents Unicode code points using one to four bytes, with ASCII characters (U+0000 to U+007F) encoded as single bytes identical to their ASCII values. It is the dominant encoding on the internet and in modern file formats, and is commonly used in embedded systems that handle human-readable text.
In practice
In many embedded applications, UTF-8 is invisible: if the firmware only handles ASCII text (menu strings, debug output, AT commands), every character fits in one byte and UTF-8 is indistinguishable from plain ASCII. This backward-compatibility property makes UTF-8 a safe default for serial debug consoles and log output even on resource-constrained 8-bit MCUs.
The complexity surfaces when the firmware must actually process multi-byte characters: parsing user input, rendering glyphs on an LCD, or measuring string length in characters rather than bytes. A standard C `strlen()` returns the byte count, not the character count. Functions like `strchr()` and pointer arithmetic over `char*` arrays are byte-oriented and will corrupt a multi-byte sequence if used naively. On bare-metal targets without a Unicode library, developers typically write or port a minimal UTF-8 decoder that walks the byte stream codepoint-by-codepoint. The blog post "Working with Strings in Embedded C++" covers practical pitfalls of string handling in embedded C++ that apply directly here.
Storage and memory costs matter on small MCUs. A 128x64 pixel monochrome display rendering a Western-European string may only need the Latin-1 subset, where every UTF-8 character is still one byte. A display needing CJK characters requires code points above U+07FF, which encode to three bytes each, and a corresponding glyph font that can easily exceed available flash. Designers often constrain the supported Unicode subset to control both code and data size.
When UTF-8 data crosses a hardware boundary -- received over UART, read from a FAT file, or fetched from an HTTP endpoint -- framing bugs can land a split multi-byte sequence across two receive buffers. Because the high bits of continuation bytes (0x80-0xBF) are distinct from lead bytes (0xC0-0xFD) and ASCII (0x00-0x7F), a UTF-8 decoder can attempt to re-synchronize by scanning forward to the next lead byte; however, a robust decoder must also verify continuation-byte structure and sequence length constraints rather than relying on lead-byte detection alone. The post "Endianness and Serial Communication" provides useful background on byte-ordering issues that interact with multi-byte data on serial links.
Frequently asked
Does using UTF-8 add overhead compared to ASCII on a small MCU?
Not if the firmware only ever processes
ASCII-range characters (U+0000 to U+007F). Those code points encode to exactly one byte in UTF-8, identical to ASCII. Overhead appears only when the code must handle multi-byte sequences: you need a decoder loop, and functions like strlen() and strchr() become character-unaware unless replaced.
How do I iterate over UTF-8 characters in C without a library?
Walk the byte array and inspect the lead byte. A byte in the range 0x00-0x7F is a complete single-byte character. 0xC0-0xDF starts a two-byte sequence; 0xE0-0xEF starts three bytes; 0xF0-0xF7 starts four bytes. The following bytes in a sequence are always 0x80-0xBF (continuation bytes). A minimal decoder extracts the code point from the non-continuation bits. This logic is roughly 20-40 lines of C and requires no
heap allocation.
Is there a byte-order (endianness) concern with UTF-8?
No. UTF-8 is defined as a byte stream with a fixed
byte order determined by the encoding rules themselves. There is no endianness ambiguity, which is one reason it became the preferred Unicode encoding for interchange over UTF-16 (which requires an agreed byte order established either by a BOM or out-of-band agreement) and UTF-32.
What is a BOM (byte-order mark) and should I include one in UTF-8 output from firmware?
The UTF-8 BOM is the three-byte sequence 0xEF 0xBB 0xBF, which is the UTF-8 encoding of U+FEFF used as a stream signature. It is optional and generally discouraged in UTF-8 streams. Many parsers,
POSIX tools, and embedded host-side scripts will misinterpret or choke on a leading BOM. Omit it unless a specific host-side tool explicitly requires it.
Can a corrupt or truncated UTF-8 byte stream cause a buffer overrun in a naive decoder?
Yes. A naive decoder that reads N continuation bytes based on the lead byte, without bounds-checking the buffer, will read past the end of a truncated sequence. Always validate that the required continuation bytes exist within the buffer before consuming them, and reject or skip sequences with invalid continuation bytes (values outside 0x80-0xBF).
Differentiators vs similar concepts
UTF-8 is one of several Unicode transfer encodings. UTF-16 encodes most common code points as two bytes (with surrogate pairs for code points above U+FFFF) and requires a defined
byte order via a BOM or out-of-band agreement, adding endianness complexity on
little-endian MCUs. UTF-32 uses a fixed four bytes per code point, making indexing trivial but quadrupling storage for
ASCII-heavy text. Latin-1 (ISO 8859-1) covers only 256 code points and is not Unicode; its byte values 0x00-0xFF overlap with the first 256 Unicode code points, but the raw bytes 0x80-0xFF are not valid UTF-8 on their own (the corresponding code points U+0080-U+00FF are representable in UTF-8 as two-byte sequences). ASCII is a strict 7-bit subset of UTF-8 and is fully compatible with it.
