Simple integration of RabbitMQ and Spring Boot

A translation of the article was prepared ahead of the start of the course. “Developer on the Spring Framework”.


Hello everyone!

I would like to share with you open source library, which facilitates the integration of RabbitMQ with applications on Spring Boot. In addition, this library offers a new, improved concept of retries (compared to the standard Spring AMQP approach).

Simplifies? But how?

Autoconfiguration

Suppose we want to implement interaction (for example, using SSL) with RabbitMQ using Spring AMQP. We need to create some beans, such as ConnectionFactory, RabbitAdmin, RabbitTemplate and AbstractRabbitListenerContainerFactory.

But imagine that there are several virtual hosts in RabbitMQ. By default, in Spring AMQP, you need to create such an ecosystem for each virtual host with manual configuration of beans RabbitTemplate and RabbitListener.

As you can see, all this configuration takes time and can be a headache.

Aware of this problem, the proposed library automatically configures all these beans for you. The only thing you need to do is define the configuration in application.propertiesand magic will happen!

Wow, but what does the configuration look like in application.properties?

Everything is very simple. Firstly, there is a prefix for the properties – this spring.rabbitmq.custom.

After that, you must specify the name of the event. This is a very important part, since all the settings that the library makes are based on this name.
After the event name, you can specify the property and value. All properties are described in documentation.

So, the template is as follows:

spring.rabbitmq.custom.<YOUR_EVENT>.<PROPERTY> =<VALUE>

The following is an example configuration for two different connections.

spring.rabbitmq.custom.some-event.host=localhost
spring.rabbitmq.custom.some-event.port=5672
spring.rabbitmq.custom.some-event.ttlRetryMessage=5000
spring.rabbitmq.custom.some-event.maxRetriesAttempts=5
spring.rabbitmq.custom.some-event.ttlMultiply=2
spring.rabbitmq.custom.some-event.queueRoutingKey=ROUTING.KEY.TEST
spring.rabbitmq.custom.some-event.exchange=ex.custom.direct
spring.rabbitmq.custom.some-event.exchangeType=direct
spring.rabbitmq.custom.some-event.queue=queue.custom.test
spring.rabbitmq.custom.some-event.autoCreate=true
spring.rabbitmq.custom.some-event.concurrentConsumers=1
spring.rabbitmq.custom.some-event.maxConcurrentConsumers=1
spring.rabbitmq.custom.some-event.virtualHost=tradeshift
spring.rabbitmq.custom.some-event.primary=true
spring.rabbitmq.custom.some-event.sslConnection=true
spring.rabbitmq.custom.some-event.tlsKeystoreLocation=file:///etc/tradeshift/your-service/tls-keystore.pkcs12
spring.rabbitmq.custom.some-event.tlsKeystorePassword=${RABBITMQ_PASS_CERT}

spring.rabbitmq.custom.another-event.host=localhost
spring.rabbitmq.custom.another-event.port=5672
spring.rabbitmq.custom.another-event.ttlRetryMessage=5000
spring.rabbitmq.custom.another-event.maxRetriesAttempts=5
spring.rabbitmq.custom.another-event.queueRoutingKey=TEST.QUEUE
spring.rabbitmq.custom.another-event.exchange=ex_test_1
spring.rabbitmq.custom.another-event.exchangeType=direct
spring.rabbitmq.custom.another-event.queue=queue_test_1
spring.rabbitmq.custom.another-event.autoCreate=true
spring.rabbitmq.custom.another-event.concurrentConsumers=1
spring.rabbitmq.custom.another-event.maxConcurrentConsumers=1
spring.rabbitmq.custom.another-event.username=guest
spring.rabbitmq.custom.another-event.password=${RABBITMQ_PASS}

As you can see, we have not written a single line of code !!!

Good, but you were saying something about the new replay strategy, right?

Yes my friend said!

But before explaining this new strategy, let's take a look at RabbitMQ and Spring AMQP's default behavior.
By default, RabbitMQ does not provide retry processing that would allow you to control the entire message life cycle.

For example, before RabbitMQ 3.8 there was no property in the message header to control the number of retries made.

RabbitMQ default behavior:

  • If you have not identified time-to-live (TTL, lifetime), then RabbitMQ will constantly try to queue your message.
  • If you have defined TTL but not defined dlx, then after TTL the message will be deleted from the queue and you will lose it.
  • If you have defined TTL and dlxthen after TTL the message will be sent to exchangerdefined in dlx.


RabbitMQ default behavior

But what if we want an increasing TTL (for example, in case of unstable operation) and control the number of retries?

Great question!
It's time to explain how Spring AQMP and Spring Rabbit Tuning library work!

Retry strategies

Spring AMQP by default

Using Spring AMQP by default You can define retry settings using the properties below.
But this approach has problems. By default, Spring AMQP will block your queue when you try to deliver a message again.
This problem can be solved in a roundabout way: use parallelism. But in this way we load the JVM, and this is not the best approach. If you have five problematic messages (as in the example), then a bottleneck will again arise, and we still need to manually determine the beans in @Configuration - a bin for each connection and container.

spring.rabbitmq.listener.simple.retry.enabled=true
spring.rabbitmq.listener.simple.retry.initial-interval=2000
spring.rabbitmq.listener.simple.retry.max-attempts=5
spring.rabbitmq.listener.simple.retry.multiplier=2
spring.rabbitmq.listener.simple.max-concurrency=5
spring.rabbitmq.listener.simple.concurrency=5

Spring RabbitMQ Tuning

This library takes a different approach with a separate queue for retries. This way we can control TTL using the expiration message parameter, and the x-death parameter to control the number of retries.

But how?

We use the dlx concept in the retry queue to resend the message to the main queue. Thus, we have access to the x-death parameter and we can programmatically determine the lifetime of the message.
Note. Since this library is an extension of Spring AMQP, you can use the default retry strategy, and use this library only for automatic configuration of beans.


Spring RabbitMQ Tuning Retry

Can I use your retry strategy without autoconfiguration? I already have many beans, and I don’t have time to rewrite them.

We know that the world is not perfect, and for this situation, we have a flag that allows you to disable auto-configuration and use only our approach to repetition processing and other advantages, such as bin management.

You can disable autoconfiguration using the following property:

spring.rabbitmq.enable.custom.autoconfiguration=false

But I have two different connections, what should I do in this case?

If you have more than one connection and you want to disable autoconfiguration, then you need to specify the bean name RabbitTemplate for each connection. This is described here.

spring.rabbitmq.custom.<YOUR_EVENT> .rabbitTemplateBeanName = <RABBITTEMPLATE_BEAN_NAME>

You can still use our bean RabbitTemplateHandler to facilitate work with Spring AMQP.

Very well! So, I want to use this. How can i do this? Do you have an example or documentation?

Yes!

We have an example project in this repositorywhere you can see how to use this library for publishers and subscribers.
But it is so simple that I will set an example here!

Publisher

There is a class in the library RabbitTemplateHandler and it is very easy to use. Need to call a method getRabbitTemplate and pass the virtual host as a parameter to get the bean RabbitTemplate. After that, you can call the convertAndSend method and pass it in the exchange and routing key parameters.

Note: exchange and routing key can be obtained using the annotation Value

Example:

@Value("${spring.rabbitmq.custom.some-event.exchange}")
private String exchangeSomeEvent;

@Value("${spring.rabbitmq.custom.some-event.queueRoutingKey}")
private String routingKeySomeEvent;

@Autowired
private final RabbitTemplateHandler rabbitTemplateHandler;

public void sendMessage(final String message) {
   rabbitTemplateHandler.getRabbitTemplate("some-event").convertAndSend(exchangeSomeEvent, routingKeySomeEvent, message);
}

Subscriber

Subscribers are also very simple. The only thing you need to do is add an annotation to the method RabbitListener (Spring AMQP annotation by default) and pass name containerFactory.

How to find out the correct containerFactory name for a virtual host?

You do not need to know it, just pass the name of the event, and the library will do all the magic for you! This library also has the ability to include retries and dlq, which are recommended.

Turning them on is very simple. You need to add annotation to the method EnableRabbitRetryAndDlq and pass the name of the property as an argument.

Note: You can also specify which exceptions to handle when retrying and dlq. Processed by default Exception.class, which means handling all exceptions.

Example:

@RabbitListener(containerFactory = "some-event", queues = "${spring.rabbitmq.custom.some-event.queue}")
@EnableRabbitRetryAndDlq(event = "some-event", exceptions = { IllegalArgumentException.class, RuntimeException.class })
public void onMessage(Message message) {
   ...
}

It's all! I hope you enjoy this library as much as we do!
And, most importantly: feel free to contribute or send feedback!

Special thanks to

Andre Luis Gomes
Rogerio bueno
Leonardo Ferreira


Sign up for a free lesson.


Similar Posts

Leave a Reply