Running a Single Consumer

So far, we have seen Kafka consumers run as part of a consumer group that automatically handles rebalances and assignment of partitions. However, there may be scenarios where you want to run a single consumer that reads records from all or just a few of the partitions of a topic. In this case, there’s no need for a consumer group or rebalancing. It is sufficient for a single consumer to start consuming from the topic partitions and commit offsets as it deems fit.

The code widget below demonstrates how to implement a single consumer. Note that in the properties we still specify the property group.id, as it is a mandatory property even though the consumer group will consist of a single consumer. Also, note we don’t use the subscribe API on the consumer object. Rather, we use the assign API to read all the partitions of the topic. Go through the comments in the listing below to understand the changes we just discussed.