Commit Log

Kafka is based on the concept of a commit log. Understanding the design and concept of the commit log will help us understand Kafka better. A commit log is a sequence of records where each record has its own unique identifier. There are some characteristics of a commit log that differentiate it from other kinds of logs and storage:

• Records can only be appended at the end of a commit log.
• Records are immutable (can’t be modified).
• A commit log is always read from left to right. The record offsets can be used to specify the start and end of a read.

Commit logs have been used in software for a long time and serve as a reliable record of what has happened in a system and in what order. Databases and caches often used commit logs to reconstruct the state of the system from a crash or for optimizing performance. Commit logs are also known as write ahead logs.