Spring Cloud Config and updating beans at runtime

There was a need for certain service components to be able to pull up the updated configuration and work based on this configuration, i.e. the application configuration changes after the service is started. I will conduct a short review of the approaches that I found in relation to such a task and what I stopped at.

The project uses spring boot 2.6.4 and kotlin 1.5.31. Also, spring cloud config server is used to configure services, where Git + Vault is used as a backend.

Spring Cloud Config Server

For testing approaches, it is easier to use the file system as the backend of Spring Cloud Config Server:

#docker-compose.yml
version: '2'
services:
 config-server-env:
   container_name: config-server-env
   image: hyness/spring-cloud-config-server:2.2
   ports:
     - "8888:8888"
   environment:
     SPRING_PROFILES_ACTIVE: native
   volumes:
     - ./config:/config

Let’s place the configuration for the refresh_app application with the dev profile in the ./config directory:

#config/refresh_app-dev.yml
refresh:
  property1: 1

After running this configuration, you can check what configuration the Spring Cloud Config Server gives for the refresh_app test service:

 >curl http://localhost:8888/refresh_app/dev
{"name":"refresh_app","profiles":["dev"],"label":null,"version":null,"state":null,"propertySources":[{"name":"file:config/refresh_app-dev.yml","source":{"refresh.property1":1}}]}

Spring Cloud Config Client (refresh_app service) configuration

//build.gradle
..............
implementation "org.springframework.boot:spring-boot-starter"
implementation "org.springframework.cloud:spring-cloud-starter-config"
implementation "org.springframework.boot:spring-boot-starter-actuator"
implementation "org.springframework.boot:spring-boot-starter-web"
..............
#application.properties
spring.config.import=optional:configserver:http://localhost:8888
spring.application.name=refresh_app
server.port=8080
management.endpoints.web.exposure.include=*

Updating the configuration and synchronizing with the service

To update the refresh_app configuration on the Spring Cloud Config Server side in our example, simply update the config/refresh_app-dev.yml file. To make sure that our server is returning an already updated configuration, you can issue a GET request again /refresh_app/dev to spring cloud config server.

Next, the actuator endpoints of the refresh_app service will be used:

  • POST /actuator/refresh

    Calling this endpoint queries the current configuration from the Spring Cloud Server. Checks the received configuration against the current application configuration and generates a list of properties that have changed and updates the service’s Environment. The Response body of this request contains a Set of changed properties. Also inside the application, an EnvironmentChangeEvent event is generated, which also contains a Set of updates, you can subscribe to it.

  • GET /actuator/env

    The Response body displays the service’s current Environment. With this call, you can make sure that the updated configuration has been pulled up.

To enable actuator endpoints for a service:

How to make application components use the updated configuration

Consider the simplest example: there is a certain service that, in response to a request, generates a response based on the refresh.property1 value. We implement this simple logic in three different ways.

1) Environment

@RestController
class Rest1(val applicationContext: ApplicationContext) {
   @GetMapping("/test1")
   fun test(): ResponseEntity<String> = 
    ResponseEntity.ok(applicationContext.environment["refresh.property1"])
}

2) @ConfigurationProperties

@ConfigurationProperties("refresh")
class TestProperties {
   lateinit var property1: String
}
@RestController
class Rest2(val testBean: TestProperties ) {
   @GetMapping("/test2")
   fun test(): ResponseEntity<String> = ResponseEntity.ok(testBean.property1)
}

I want to draw attention to the peculiarity of the work of kotlin and spring. The following constructs will be correctly initialized at application startup time, but will not be updated at runtime:

@ConstructorBinding 
@ConfigurationProperties("refresh") 
data class TestBean(val property1: String)

@ConstructorBinding 
@ConfigurationProperties("refresh") 
data class TestBean(var property1: String)

3) @RefreshScope + @Value

@RefreshScope
@RestController
class Rest3(@Value("\${refresh.property1}") val property1: String) {
   @GetMapping("/test3")
   fun test(): ResponseEntity<String> = ResponseEntity.ok(property1)
}

All three of the above implementations in the response will return the actual value of refresh.property1 after syncing with Spring Cloud Config Server using the POST /actuator/refresh call.

In this case, the logic of the service does not require any logic to update the beans, but in some cases a more complex approach is required.

Reinitializing Existing Beans

The previous section considered the simplest case of changing the application logic after updating the configuration. But in some cases, after updating some properties, the application component needs to be reinitialized.

For example, there is a bean that is a wrapper over, say, KafkaConsumer. The @PostConstruct method initializes and starts the Consumer, the @PreDestroy method stops and deinitializes the Consumer. In this case, I would like this bean to react to a change in the Environment and, if the properties associated with it have changed, be reinitialized. What does reinitialization mean in this context – for example, calling the @Predestroy method and calling the @PostConstruct method, given the already updated configuration.

As mentioned earlier, the result of a request to the POST /actuator/refresh service is the generation of an EnvironmentChangeEvent event with a list of changed properties, to which you can subscribe and call certain logic.

Those. you can implement your own Bean Post Processor calling the reinitialization process of certain service components if certain properties have been updated. I have introduced new @Refreshable and @Refresh annotations:

@Target(ElementType.METHOD, ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
annotation class Refreshable(val property: String)

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
annotation class Refresh

@Refreshable sets the property on which the bean depends and when it changes, it is updated. @Refresh – for the method that will be executed if the properties associated with the bean are updated.

Initially, I wanted to get by with one @Refreshable annotation and, in the case of an update, deinitialize and initialize the bean as the spring context does, but it came to an understanding that many factors need to be taken into account and it is better to give the opportunity to explicitly specify which method will be executed during reinitialization using @Refresh.

Bean Post Processor Implementation:

class RefreshBeanPostProcessor(private val applicationContext: GenericApplicationContext) : BeanPostProcessor {

   private val refreshPropertyTasks = mutableListOf<RefreshPropertyTask>()

   override fun postProcessBeforeInitialization(bean: Any, beanName: String): Any {
       (getRefreshablePropertyByBean(bean) ?: getRefreshablePropertyByBeanDefinition(beanName))
           ?.let { property ->
               bean.javaClass.methods
                   .firstOrNull { it.getAnnotation(Refresh::class.java) != null }
                   ?.let { method -> 
                     refreshPropertyTasks.add(RefreshPropertyTask(property) 
                                              { method.invoke(bean) }) }
           }
       return bean
   }

   @PostConstruct
   fun init() {
       applicationContext.addApplicationListener { event ->
           if (event is EnvironmentChangeEvent) {
               onApplicationEvent(event)
           }
       }
   }

   private fun onApplicationEvent(event: EnvironmentChangeEvent) {
       refreshPropertyTasks
           .filter { task -> event.keys.firstOrNull { key -> key.startsWith(task.property) } != null }
           .forEach { task ->
               try {
                   log.info("Update task ${task.property}")
                   task.action()
               } catch (ex: Exception) {
                   log.error("Error for task ${task.property}", ex)
               }
           }
   }

   private data class RefreshPropertyTask(val property: String, val action: () -> Unit)

   private fun getRefreshablePropertyByBean(bean: Any): String? =
       bean.javaClass.getAnnotation(Refreshable::class.java)?.property

   private fun getRefreshablePropertyByBeanDefinition(beanName: String): String? =
       applicationContext.beanFactory
           .let { kotlin.runCatching { it.getBeanDefinition(beanName) }.getOrNull() }
           ?.source
           ?.takeIf { it is AnnotatedTypeMetadata }
           ?.let { it as AnnotatedTypeMetadata }
           ?.getAnnotationAttributes(Refreshable::class.java.name)
           ?.let { it[Refreshable::property.name] as String? }
}

It is important to mention that it is necessary to add @DependsOn(“configurationPropertiesRebinder”) for this post processor. The configurationPropertiesRebinder is the bean that actually updates the @ConfigurationProperties classes. It implements the ApplicationListener interface in the same way as the implemented Bean Post Processor, and if, when initializing the ApplicationListener, it will be in the context list after ours, then at the time of our reinitialization logic, the @ConfigurationProperties classes will not be updated yet and our reinitialization will occur with irrelevant values ​​if it depends on @ConfigurationProperties classes.

@DependsOn("configurationPropertiesRebinder")
@Bean
fun refreshBeanPostProcessor(ctx: GenericApplicationContext) = 
  RefreshBeanPostProcessor(ctx)

This guarantees us that the ApplicationListener implemented inside the Bean Post Processor will be listed after the configurationPropertiesRebinder and accordingly will be executed after.

Examples of using:

@ConfigurationProperties("refresh")
class TestProperties{
   lateinit var property1: String
   lateinit var property2: String
}

@Refreshable("refresh")
@Service
class Bean1(testProperties: TestProperties){
  
   @PostConstruct fun init(){}
   @PreDestroy fun destroy(){}
  
   @Refresh
   fun refresh(){
       init()
       destroy()
   }
}

class Bean2(testProperties: TestProperties){

   @PostConstruct fun init(){}
   @PreDestroy fun destroy(){}

   @Refresh
   fun refresh(){
       init()
       destroy()
   }
}

@Configuration
class TestConfiguration{

   @Refreshable("refresh")
   @Bean
   fun bean2(testProperties: TestProperties) = Bean2(testProperties)
}

Auto-update Environment Service

Spring Cloud Config Server knows nothing about clients and to update the configuration on each instance of our service, you need to call POST /actuator/refresh.

One option to make service updates easier is to use the Spring Cloud Bus. It is necessary to set up integration on services and Spring Cloud Config Server through adding a dependency and setting up a configuration. Kafka, RabbitMQ, etc. can be used for integration. After setting up, by calling GET /monitor Spring Cloud Config Server, all services will be updated via messaging through the Kafka topic. If you use a system that supports webhooks as the backend for Spring Cloud Config Server, you can set up a call to GET /monitor via webhook. I have decided not to dwell on this decision yet, because. while it seems that such additional integration complicates the support and configuration of the system.

At the moment, I decided to make auto-update inside each service. It looks like a periodic request should not add any significant load on the system, even if the total of all instances of all services is about 100 in total.

Option to implement a periodic request to update the Environment:

class RefreshEventPublisherScheduler(
   private val refreshSchedulerProperties: RefreshSchedulerProperties,
   private val applicationEventPublisher: ApplicationEventPublisher
) {

   companion object : Log()

   private val executor: ScheduledExecutorService = Executors.newSingleThreadScheduledExecutor()

   @PostConstruct
   fun init() {
       schedule()
   }

   private fun schedule() {
       executor.schedule(
           {
               if (refreshSchedulerProperties.enabled) {
                   publishRefreshEvent()
               }
               schedule()
           },
           refreshSchedulerProperties.interval.toMillis(),
           TimeUnit.MILLISECONDS
       )
   }

   private fun publishRefreshEvent() {
       applicationEventPublisher.publishEvent(
           RefreshEvent(
               this,
               "Refresh event",
               "Refresh scope"
           )
       )
   }
}

Those. each client is scheduled to call the publishRefreshEvent() method, which corresponds to a POST /actuator/refresh call.

The implemented Bean Post Processor and RefreshEventPublisherScheduler can be included in the starter and used in services for auto-updating and reinitializing beans.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *