Spring Boot Kafka Batch Listener Example, easy in 15 minutes

Today, I introduce a Spring Boot Kafka Batch Listener Example and demonstrate how to implement Kafka producer with consumer Batch Listener. I make sure that this tutorial is useful with your real projects. Let’ do together.

Spring Boot Kafka Batch Listener Example

Project Setup

  • Spring Kafka: 2.3.4.RELEASE
  • Spring Boot: 2.2.2.RELEASE
  • Apache Kafka: kafka_2.13-2.4.0
  • Maven: 3.5

Project Structure

Spring Boot Kafka Batch Listener Example

Maven Dependencies

At least, we must have these dependencies: spring-boot-starter, spring-kafka in pom.xml file. Note, you can change spring-kafka version for your own needs.

Configure Spring Kafka Application with application.yml

Some properties that need to be explicitly set in the application.yml file, e.g host, port, topic name:

Create Kafka Producer and Consumer

If you want to understand deeply how to create Producer and Consumer with configuration, please the post Spring Boot Kafka Producer Consumer Configuration  or You can also create Spring Boot Kafka Producer and Consumer without configuration, let check out the post Spring Boot Apache Kafka Example. Here we reuse Sender and SenderConfig from the post Spring Boot Kafka Producer Consumer Configuration.

Configuring a Batch Listener

We maybe need to do these steps:

  1. Enable the Batch Listener property of ConcurrentKafkaListenerContainerFactory using the method setBatchListener(true).
  2. Create a BatchErrorHandler using the method setBatchErrorHandler(new BatchLoggingErrorHandler()). You may create your own Batch Error Handler.
  3. Set an upper limit for the batch size by setting the ConsumerConfig.MAX_POLL_RECORDS_CONFIG to a value that you need. Here we are setting that value to 4.

You can have a look in the full configuration of ReceiverConfig

Receive Kafka Messages using a Batch Listener

We are receiving batch messages so the method receive() must accept a list of messages, and certainly, this method can accept a list of headers and a list of offsets. Let’s have a look Receiver below:

 

Spring Boot Application

To demo our Spring Boot Kafka Batch Listener Example then we create a simple Spring Boot Application below.

Running Spring Boot Application

Running Spring Boot Application normally and check out the output. Just remind that Sender that have sent a total of 9 messages, then Receiver will be consumed by three batches. You will see that in the output below.

That’s all about Spring Boot Kafka Batch Listener Example. Now you can try to do your own practices and don’t forget to download the complete source code of Spring Boot Kafka Batch Listener Example below.

Download the complete source code

spring-kafka-batchlistener-example.zip (82 downloads)

References

Spring Boot Kafka Producer Consumer Configuration
Spring Boot Apache Kafka Example
Spring Kafka Client Compatability

Please share it if you found this useful
Hide Buttons