Testing Kafka Streams Topologies

Every time we add or change the logic in our Kafka Streams topology, we should verify that our changes have the correct outcomes. This verification can be done by:

1. Starting a Kafka cluster

2. Running the entire application

3. Publishing input messages using the producer CLI

4. Reading the output message using the consumer CLI

While we should do this at least once or twice to verify all the parts of the system are playing correctly together, performing this for every change we make will become very cumbersome. If we want to automatically ensure that our topology behaves as we intended it to, we should run unit tests that test the topology’s behavior.

Luckily, we can do it quite easily! Kafka Streams has a test utilities library which comes with a TopolgyTestDriver class that allows us to quickly and conveniently test our topologies.

Adding the test utils dependency

Add the line below to the build.gradle file in the dependencies closure (under line 19, for example):