0% completed

All LessonsFree Lessons (5)

Basics

Characteristics of Distributed Systems

Messaging Patterns

Components of Kafka

Partition Rebalancer

Kafka Producer

Producer Sending Messages Producer Configurations Producer Serialization

Kafka Consumer

Consumer and Consumer Groups Kafka Consumer Consumer Configurations Commits and Offsets Offset Commit Configuration Handling Rebalances Stopping a Consumer Consumer Deserializer Running a Single Consumer

Kafka Internals

Replication Kafka Controller Request Processing Partition Allocation Data Storage Reliability in Kafka Reliable Producers and Consumers

Conclusion

Kafka vs Other Messaging Systems

Appendix

Zookeeper: Intro Zookeeper: Example Zookeeper: Practical

Reference: Replication

Introduction Single Leader Replication Asynchronous VS Synchronous Replication Followers in Log Replication Log Replication Issues in Single Leader Replication Multi Leader Replication Issues in Multi Leader Replication Multi-Leader Topologies Leaderless Replication R + W > N Quorum Variations Concurrent Writes Video Streaming Queue a Concurrent Writes Example

Reference: Partitioning

Introduction Partitioning Schemes Secondary Indexes Number of Partitions

Reference: Transactions

Introduction to Transactions More on Transactions Isolation Levels Read Skew and Snapshot Isolation Concurrent Writes and Lost Updates Write Skew Serializability Two Phase Locking Serializable Snapshot Isolation

Reference: Issues in Distributed Systems

Introduction Networking Agreeing On Time Working with Time Issues

Building Scalable Data Pipelines with Kafka/

...

/

Commit Log

Commit Log

This lesson delves into the concept of a commit log and how it relates to Kafka.

We'll cover the following...

Kafka is based on the concept of a commit log. Understanding the design and concept of the commit log will help us understand Kafka better. A commit log is a sequence of records where each record has its own unique identifier. There are some characteristics of a commit log that differentiate it from other kinds of logs and storage:

Records can only be appended at the end of a commit log.
Records are immutable (can’t be modified).

...