Kafka Consumer
Explore how Kafka consumers operate with essential properties like deserializers and group IDs. Learn to manage polling loops for message processing and understand consumer group coordination and partition rebalancing. This lesson helps you implement efficient Kafka consumers for scalable data pipelines.
We'll cover the following...
Kafka consumers are instances of the class KafkaConsumer, which is a parallel of the KafkaProducer class used to instantiate producers. The KafkaConsumer class requires three compulsory properties to be provided: the location of the servers bootstrap.servers, the key deserializer key.deserializer, and the value deserializer value.deserializer. The deserializers convert byte arrays into Java objects.
When creating a KafkaConsumer we can also specify a consumer group id using ...