Encoding Formats Guide: UTF-8, Base64, and More

What are Encoding Formats?

Encoding formats define how characters and data are represented as bytes. Understanding encoding is essential for handling text across different systems, languages, and platforms without data corruption.

UTF-8

UTF-8 is the dominant character encoding for the web. It uses 1 to 4 bytes per character and supports every Unicode character. It is backward-compatible with ASCII, making it the default choice for most applications.

const text = "Hello, 世界 🌍";
const encoded = new TextEncoder().encode(text);
console.log(encoded); // Uint8Array with UTF-8 bytes

ASCII

ASCII (American Standard Code for Information Interchange) uses 7 bits per character and supports 128 characters (0-127). It covers basic English letters, digits, and punctuation but lacks support for international characters.

Base64 Encoding

Base64 converts binary data into a text representation using 64 ASCII characters. It is commonly used for embedding binary data in JSON, emails, and URLs.

const data = "Hello, World!";
const base64 = btoa(data);
console.log(base64); // "SGVsbG8sIFdvcmxkIQ=="

URL Encoding

URL encoding (percent encoding) converts special characters into a format safe for URLs. Spaces become %20, and non-ASCII characters are percent-encoded.

const url = "https://example.com/search?q=hello world";
const encoded = encodeURI(url);
// "https://example.com/search?q=hello%20world"

HTML Entities

HTML entities represent special characters using named or numeric codes. For example, < represents <, > represents >, and & represents &.

BOM (Byte Order Mark)

The BOM is a special Unicode character (U+FEFF) placed at the beginning of a text stream to indicate byte order. UTF-8 BOM is optional but can cause issues in some systems.

Character Sets Comparison

Encoding	Bytes per Char	Max Characters	Use Case
ASCII	1 (7-bit)	128	English only
UTF-8	1-4	1,112,064	Universal
UTF-16	2-4	1,112,064	Java, Windows
Base64	N/A	N/A	Binary to text