Sending a message to multiple Kafka listeners at the same time

This article describes a way to send a message to multiple Kafka listeners at the same time. Multiple listeners will receive the same message, from the same message sender, in this solution implementation.

The logic is implemented using Java, Spring and Kafka.

Plan

1. Create Kafka listeners

2. Create Kafka Message Sender

3. Create a testing solution

4. Conduct testing

5. Create a list of definitions for the mechanisms used in the solution

1. Create Kafka listeners

Let's create 2 listeners via annotation KafkaListener. These listeners must have the same topic name(topic) and different group names.

@Service
public class KafkaTester {

    //Listener 1
    @KafkaListener(id = "id1",
            groupId = "group-one",
            topics = "topic-one")
    public void listenServiceCall(@Payload String message) {
        //Logging
        System.out.println("GROUP ONE MESSAGE " + message);
    }

    //Listener 2
    @KafkaListener(id = "id2",
            groupId = "group-two",
            topics = "topic-one")
    public void listenServiceCall2(@Payload String message) {
        //Logging
        System.out.println("GROUP TWO MESSAGE " + message);
    }
}

Logging the message in listeners is necessary to test the solution.

2. Create a Kafka message sender

Let's create a message sender using the class KafkaTemplate.

@Service
public class KafkaTester {

    @Autowired
    private  KafkaTemplate<String, String> kafkaTemplate;

    //... LISTENERS FROM PREVIOUS EXAMPLE
}

The KafkaTemplate object is embedded in the KafkTester service via an annotation Autowired.

3. Create a testing solution

Let's create a method that will be activated at certain time intervals. Let's call the method scheduledSend(). Next, we’ll write the logic for sending a message via Kafka, within the method scheduledSend().

To implement a method that will be turned on periodically(scheduledSend()), the Scheduled annotation is used. The period for enabling the method is set to 10 seconds.

@Service
public class KafkaScheduler {

    @Autowired
    private  KafkaTemplate<String, String> kafkaTemplate;

    //10 seconds period
    @Scheduled(cron = "*/10 * * * * *")
    public void send() {
        kafkaTemplate.send("topic-one", "kafkaMessage " + new Date());
        System.out.println("MESSAGE WAS SENT");
    }

    @KafkaListener(id = "id1",
            groupId = "group-one",
            topics = "topic-one")
    public void listenServiceCall(@Payload String message) {
        System.out.println("GROUP ONE MESSAGE " + message);
    }

    @KafkaListener(id = "id2",
            groupId = "group-two",
            topics = "topic-one")
    public void listenServiceCall2(@Payload String message) {
        System.out.println("GROUP TWO MESSAGE " + message);
    }
}

In this code you can see that it is necessary to send a message indicating the same topic name as the listeners. In this case, listeners will receive the same message.

4. Conduct testing

1. Run an application using class code KafkaScheduler.

2. Receive a program log with the same time in messages, accurate to the second.

SENT MESSAGE
GROUP ONE MESSAGE kafkaMessage Tue Sep 12 15:17:40 EDT 2023
GROUP TWO MESSAGE kafkaMessage Tue Sep 12 15:17:40 EDT 2023
SENT MESSAGE
GROUP ONE MESSAGE kafkaMessage Tue Sep 12 15:17:50 EDT 2023
GROUP TWO MESSAGE kafkaMessage Tue Sep 12 15:17:50 EDT 2023

5. List of definitions

Topic name – a label for grouping messages.

Group ID (groupId) – a label for grouping listeners.