Building a multi-module Gradle project in Gitlab CI

What could be easier? Writing a build commandgradle clean buildand you’re done. At first glance, everything is really so, and it will take a little time. But over time, the code base and, accordingly, the number of tests (well, I really hope so) will grow, you will not have time to come to your senses as the assembly will take you 10 or more minutes

Let’s think about what can help us with this? And for this, we analyze the main steps of this process:

  1. Resolving dependencies on external libraries

  2. Compiling Java code

  3. Test run

We break the project into modules

And also remember (or learn) about such a gradle feature as build cache. How can she help us? Indeed, in fact, if we make any change to the java code, this will cause the code to be recompiled and all tests to run. In this case, the modularity of the project can help us. If our code is quite strongly segmented (for example, by domain models), and often as part of the execution of tasks we make changes only to some part of the project, then there is no need to recompile the entire project and run absolutely all tests.

For example, we have some kind of online store project, it has several main domains:

For example, if you make changes to

  • account module – this will cause the account and orders module (as a dependency) to be recompiled and tests run in them. The product module does not need to be recompiled and tests run in it, because code has not changed

  • orders module – this will cause only that module to be recompiled and its tests to be executed. The account and product modules remain untouched – there is already much more profit here!

So, our startup script already looks like this: gradle clean build --build-cache

Run on CI

Okay, we checked it locally – everything is super, re-running the command works out in seconds, I send it all to Gitlab CI. And what do we see in the end? And there is no profit. Of course, each job is executed in an isolated container, and there are no caches in it, all dependencies are downloaded from Maven Cental each time, the project is compiled each time, and all tests are executed each time.

You need to somehow transfer “artifacts” between assemblies. Here we can help Gitlab Cache. Okay, we have the tool, now we need to decide what we want to transfer.

Caching Dependencies

Downloaded dependencies are by default stored in ${USER_HOME}/.gradle

build:
  stage: build
  image: gradle:7.2-jdk17
  before_script:
    - export GRADLE_USER_HOME=`pwd`/.gradle_home
  script:
    - gradle clean build check --stacktrace --info --build-cache
  cache:
    - key: dep-cache
      paths:
        - .gradle_home

Now dependencies will not be downloaded each time, but will be cached (most likely in s3, depending on how your Gitlab is configured). This, although not free, like local storage, is still faster than going to an external network.

Caching build results

Next, we need to somehow cache the results of project compilation and test results. Artifacts are by default in the folder build in each module. Registering each folder manually is not an option, we are programmers. Let’s change the location of buildDir in each model. To do this, we will make changes to the root build.gradle

ext {
    set('rootProjectDir', "${projectDir}")
}
allprojects {
    buildDir = "${rootProjectDir}/.build/${project.name}/build"
}

and add folder .build from the root of the project under the cache.

cache:
  - key: build-cache
    paths:
      - .build

At this step, we should already get some profit from the work done.

A little about TestContainers

Let’s look at another case – for example, using TestContainers to dockerize the environment in tests. There is a point here – that when we are inside the container, when we run tests, we will load images for internal containers every time. If you have a corporate docker hub (caching proxy), then we can suggest a way to download faster from it:

build:
  stage: build
  image: gradle:7.2-jdk17
  before_script:
    - export GRADLE_USER_HOME=`pwd`/.gradle_home
    - export TESTCONTAINERS_RYUK_CONTAINER_IMAGE=my-docker-hub.io/testcontainers/ryuk:0.3.3
  script:
    - gradle clean build check --stacktrace --info --build-cache

Or we write the code with pens.

PS That it is not necessary to beat the monolith into modules, but it is not worth writing comments on microservices – the goal was to show the possibility of the buildDir configuration

Similar Posts

Leave a Reply