Building a multi-module Gradle project in Gitlab CI
What could be easier? Writing a build commandgradle clean build
and you’re done. At first glance, everything is really so, and it will take a little time. But over time, the code base and, accordingly, the number of tests (well, I really hope so) will grow, you will not have time to come to your senses as the assembly will take you 10 or more minutes
Let’s think about what can help us with this? And for this, we analyze the main steps of this process:
-
Resolving dependencies on external libraries
-
Compiling Java code
-
Test run
We break the project into modules
And also remember (or learn) about such a gradle feature as build cache. How can she help us? Indeed, in fact, if we make any change to the java code, this will cause the code to be recompiled and all tests to run. In this case, the modularity of the project can help us. If our code is quite strongly segmented (for example, by domain models), and often as part of the execution of tasks we make changes only to some part of the project, then there is no need to recompile the entire project and run absolutely all tests.
For example, we have some kind of online store project, it has several main domains:
For example, if you make changes to
-
account module – this will cause the account and orders module (as a dependency) to be recompiled and tests run in them. The product module does not need to be recompiled and tests run in it, because code has not changed
-
orders module – this will cause only that module to be recompiled and its tests to be executed. The account and product modules remain untouched – there is already much more profit here!
So, our startup script already looks like this: gradle clean build --build-cache
Run on CI
Okay, we checked it locally – everything is super, re-running the command works out in seconds, I send it all to Gitlab CI. And what do we see in the end? And there is no profit. Of course, each job is executed in an isolated container, and there are no caches in it, all dependencies are downloaded from Maven Cental each time, the project is compiled each time, and all tests are executed each time.
You need to somehow transfer “artifacts” between assemblies. Here we can help Gitlab Cache. Okay, we have the tool, now we need to decide what we want to transfer.
Caching Dependencies
Downloaded dependencies are by default stored in ${USER_HOME}/.gradle
build:
stage: build
image: gradle:7.2-jdk17
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle_home
script:
- gradle clean build check --stacktrace --info --build-cache
cache:
- key: dep-cache
paths:
- .gradle_home
Now dependencies will not be downloaded each time, but will be cached (most likely in s3, depending on how your Gitlab is configured). This, although not free, like local storage, is still faster than going to an external network.
Caching build results
Next, we need to somehow cache the results of project compilation and test results. Artifacts are by default in the folder build
in each module. Registering each folder manually is not an option, we are programmers. Let’s change the location of buildDir in each model. To do this, we will make changes to the root build.gradle
ext {
set('rootProjectDir', "${projectDir}")
}
allprojects {
buildDir = "${rootProjectDir}/.build/${project.name}/build"
}
and add folder .build
from the root of the project under the cache.
cache:
- key: build-cache
paths:
- .build
At this step, we should already get some profit from the work done.
A little about TestContainers
Let’s look at another case – for example, using TestContainers to dockerize the environment in tests. There is a point here – that when we are inside the container, when we run tests, we will load images for internal containers every time. If you have a corporate docker hub (caching proxy), then we can suggest a way to download faster from it:
build:
stage: build
image: gradle:7.2-jdk17
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle_home
- export TESTCONTAINERS_RYUK_CONTAINER_IMAGE=my-docker-hub.io/testcontainers/ryuk:0.3.3
script:
- gradle clean build check --stacktrace --info --build-cache
Or we write the code with pens.
PS That it is not necessary to beat the monolith into modules, but it is not worth writing comments on microservices – the goal was to show the possibility of the buildDir configuration