Apache Kafka is an open-source messaging system initially developed at Linkedin by Kreps et al.J. Kreps, N. Narkhede, and J. Rao, “Kafka: a Distributed Messaging System for Log Processing,” NetDB’ 11, June 2012, 2011, 2011. and Wang et al.G. Wang et al., “Building a Replicated Logging System with Apache Kafka,” NetDB’ 11, June 2012, 2011, 2011. and then donated to the Apache Software Foundation.

Goal of Kafka

The primary goal of Kafka is performance, scalability, durability, and availability.

Performance

The ability to exchange messages between systems with high throughput and low latency.

Scalability

The ability to incrementally scale to bigger volumes of data by adding more nodes to the system.

Durability & availability

The ability to provide durable and available data even in the presence of node failures.

Get hands-on with 1400+ tech skills courses.