Versioned Data Migration in the DTO World

DTO. The examples will be shown in Java.

Situation

You are a back-end developer and you are developing a server application that is used by various kinds of clients (client applications, not users).

The application itself consists of just one domain – the User, which in turn consists of a numeric identifier and a phone number as a string.

Java

DTOs:

@Getter
public class UserDto {
  
  private Long id;

  @JsonProperty("phone_number")
  private String phoneNumber;
}

Controller:

@RestController
public class UserController {

  @GetMapping("/api/users/{user_id}")
  public UserDto getUser(@RequestParam("user_id") Long userId) {

    // Проверка существования пользователя и получение из БД
    // Вызов логики обработки
    // Конвертация модели в DTO

    return userDto;
  }
}
http
GET /api/users/{user_id}
Host: api.host
Accept: application/json

{
  id: 0,
  phone_number: '+70123456789'
}

At some fine point, you are told that you would like to use a different data type for the user’s phone number and return it as a number, not text as it was before.

Of course, if the command:

  • consists of magiccan deploy a server and client application at the same time without breaking anything in production;

  • can afford to maintain servers with different versions of the server application (api.host.v1, api.host.v2) and can properly support them.

In this case, you comprehended Zen and everything is fine with you)

The purpose of the article is to show how you can deploy new versions of server applications without affecting the client application, so that it can sufficiently debug interaction with new fields / data types before it is published. Well, do not put prod, of course 🙂

Migration by adding a new field

In this case, a new field with a version postfix is ​​simply added to the existing DTO and the old one is marked as deprecated (in the codebase) for further disposal. The old field continues to be supported until the moment of complete rejection of its use on the side of the client application.

Java
@Getter
public class UserDto {
    
  private Long id;

  @Deprecated
  @JsonProperty("phone_number")
  private String deprecatedPhoneNumber;

  @JsonProperty("phone_number_v1")
  private Long phoneNumber;
}
JSON
UserDto:
{
  id: 0,
  phone_number: '+70123456789',
  phone_number_v1: 70123456789
}

This option is well suited if the team is developing web applications and fast data migration is available to you (meaning that you do not need to maintain old versions of data for clients for a very long time). If the postfix will be very embarrassing, you can make another migration and simply delete it and get rid of obsolete fields in the future.

Migration through the creation of a new version of the object

In the case when clients “move” to new versions of applications for a very long time (a typical example of mobile development, moving to a newer version can take years), then sooner or later DTO will turn into a living hell if data is migrated through the field. Such code will be difficult to maintain, since there can be quite a lot of such migrations during the life of the application.

In this case, a new DTO class can be distinguished.

Java
@Getter
@Deprecated
public class UserDto {
    
  private Long id;

  @JsonProperty("phone_number")
  private String phoneNumber;
}

@Getter
public class UserDtoV1 {
    
  private Long id;

  @JsonProperty("phone_number")
  private Long phoneNumber;
}
JSON
UserDto:
{
  id: 0,
  phone_number: '+70123456789'
}

UserDtoV1:
{
  id: 0,
  phone_number: 70123456789
}

With this approach, the situation is avoided when there are so many edits in the class that difficulties with interaction and support begin, but at the same time, the number of methods that are responsible for converting data from the database to DTO increases. In this case, you can use the template Factory and for each DTO version, create an implementation that will be responsible for representing a specific version in order to give the client application a DTO version with which it can work.

What about controllers?

In the case of the controller, there are also two options for how versioning can be implemented.

There is also a good article that describes the versioning processes specifically for the API.

Migrating through the new version of the HTTP method

Java
@RestController
public class UserController {

  @Deprecated
  @GetMapping("/api/users/{user_id}")
  public UserDto depreactedGetUser(@RequestParam("user_id") Long userId) {}

  @GetMapping("/api/users/{user_id}/v1")
  public UserDtoV1 getUser(@RequestParam("user_id") Long userId) {}
}
http
GET /api/users/{user_id}
Host: api.host
Accept: application/json

{
  id: 0,
  phone_number: '+70123456789'
}

------------------------------

GET /api/users/{user_id}/v1
Host: api.host
Accept: application/json

{
  id: 0,
  phone_number: 70123456789
}

This approach is good for fast migration, but supporting a bunch of such methods can sooner or later become a headache for the developer, since the logic for converting the model to DTO between such methods will be “smeared”.

Migrating via Request Header

In this case, the header is passed accept which indicates the specific version of the JSON object that the client is ready to accept.

Example:

Java
@RestController
public class UserController {

  // Вместо возвращаемого типа Object можно создать UserBaseDto
  // в который можно вынести общие поля и использовать его как возвращаемый
  // тип метода
  @GetMapping("/api/users/{user_id}")
  public Object getUser(
    @RequestParam("user_id") Long userId,
    @RequestHeader("Accept") String acceptMimeType) {

    // Извлекаем версию конечного объекта из заголовка Accept
    // Формируем конечный объект
    
    return suitableUserDto;
  }
}
http
GET /api/users/{user_id}
Host: api.host
Accept: application/json

Response object:
{
  id: 0,
  phone_number: '+70123456789'
}

------------------------------

GET /api/users/{user_id}
Host: api.host
Accept: application/json;v=1

Response object:
{
  id: 0,
  phone_number: 70123456789
}

With this approach, it may be difficult to document the system, since the same endpoint returns not a specific view, but several at once. If your documentation system is able to describe this, then there will be no problems and further changes will be approved faster.

Conclusion

The final decision on the use of one or another approach remains with the developer. Each described option is suitable for a specific case.

PS

It would be very interesting to know how exactly your company conducts data migration)

YouTube channel of the author.

Author microblog.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *