Mastering Apache Kafka/

...

/

Producer and Consumer Applications Using Spring Kafka

Producer and Consumer Applications Using Spring Kafka

Learn how to send and receive messages to and from Kafka using Spring Kafka.

We'll cover the following...

In this lesson, we will learn to use Kafka with a Spring Boot application with the help of a couple of additional applications.

Producer application

In this section, we will use the KafkaTemplate API in a Spring Boot application to send messages to Kafka. The application consists of a REST endpoint using Spring Boot web with the spring-boot-starter-web module. The application will be packaged as a JAR file, but run as a self-contained HTTP server by using an embedded Tomcat server.

Click the “Run“ button in the widget below. This will initiate the build process and start the application. Once the application has started, you should see the Started SpringKafkaProducerApplication message.


After that, follow the steps outlined below:

package com.example.demo;

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class MessageProducer {
    
    private static final String TOPIC = "spring-kafka-demo-topic";

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendMessage(String message) {
        this.kafkaTemplate.send(TOPIC, message);
    }
}
Spring Boot Kafka: Producer application

Invoke the REST endpoint to send messages. Click the “+” button to open a new terminal tab and run these commands.

curl -i -X POST -d 'redis=keyvalue' http://localhost:8080/send
curl -i -X POST -d 'postgres=rdbms' http://localhost:8080/send
curl -i -X POST -d 'cassandra=column' http://localhost:8080/send
curl -i -X POST -d 'influxdb=timeseries' http://localhost:8080/send
curl -i -X POST -d 'mongodb=document' http://localhost:8080/send
Sending messages

For each of these commands, you should see a response similar to the following:

HTTP/1.1 200 
Content-Type: text/plain;charset=UTF-8
Content-Length: 28
Date: Mon, 31 Jul 2023 14:15:41 GMT
Sent message: redis=keyvalue

Now, you can verify that these messages were sent to Kafka.

Click the “+” button to open a new terminal tab, and enter the command to start the Kafka consumer CLI. This will wait for messages from the spring-kafka-demo-topic as shown below:

/app/confluent-7.3.1/bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic spring-kafka-demo-topic --from-beginning
Receiving messages using the Kafka consumer CLI

Code explanation

The application consists of multiple classes. Let’s go through them individually.

The KafkaProducerConfig.java class

This class is responsible for creating the Kafka producer instance using the ProducerFactory.

Here’s the step-by-step breakdown of the code:

  • Lines 3–12: We import the required packages.

  • Line 14: We use the @Configuration annotation to indicate that the Spring IoC container can use the class as a source of bean definitions.

  • Lines 17–30: ...