How to Make Code Coverage Work for You

pitest.

The last line of defense is code review. You should review the test code and logic, among other things. Human eyes can see things that automatic testing tools are unable to track. I recommend watching the report Cracking the Code Review.

What tests should not be written?

Try to avoid unit tests on mocks. Create them only in extreme cases, when it is really necessary. Very often such tests are too “small”: they are tied to implementation details, increasing the labor costs of maintaining a large number of mocks and complicating code refactoring in the future.

Try to write higher-level tests that test the functionality of your application that is primarily visible to external observers. I highly recommend using Test containers and live integrations (message queues, databases, etc.). With this approach, you write mocks for interaction with neighboring services at most.

Don't test automatically generated code: most of the time it doesn't make sense. Exclude such code from the coverage calculation. For example, for Lombok it is worth adding the config:

# lombok.config
lombok.addLombokGeneratedAnnotation = true

What tests should you add to your app?

I recommend testing various infrastructure things: metrics, swagger, health checks. For example, in a Spring Boot application, metrics may not be returned if there is no dependency on io.micrometer:micrometer-registry-prometheus or there are no required parameters in application.yml. The same with samples. Swagger can break when updating a version, etc. In fact, everything can break when updating a major version of the framework.

Until you test it, you can't guarantee that it even works.

class ActuatorEndpointTest extends TestBase {

    @LocalServerPort
    private int port;
    @LocalManagementPort
    private int actuatorPort;

    private WebTestClient actuatorClient;

    @BeforeEach
    void setUp() {
        this.actuatorClient = WebTestClient.bindToServer()
                .baseUrl("https://localhost:" + actuatorPort + "/actuator/")
                .build();
    }

    @Test
    void actuatorShouldBeRunOnSeparatePort() {
        assertThat(actuatorPort)
                .isNotEqualTo(port);
    }

    @ParameterizedTest
    @CsvSource(value = {
            "prometheus|jvm_threads_live_threads|text/plain",
            "health|{\"status\":\"UP\",\"groups\":[\"liveness\",\"readiness\"]}|application/json",
            "health/liveness|{\"status\":\"UP\"}|application/json",
            "health/readiness|{\"status\":\"UP\"}|application/json",
            "info|\"version\":|application/json"}, delimiter="|")
    void actuatorEndpointShouldReturnOk(@Nonnull final String endpointName,
                                        @Nonnull final String expectedSubstring,
                                        @Nonnull final String mediaType) {
        final var result = actuatorClient.get()
                .uri(uriBuilder -> uriBuilder
                        .path(endpointName)
                        .build())
                .accept(MediaType.valueOf(mediaType))
                .exchange()
                .expectStatus().isOk()
                .expectBody(String.class)
                .returnResult()
                .getResponseBody();
        assertThat(result)
                .contains(expectedSubstring);
    }

    @Test
    void swaggerUiEndpointShouldReturnFound() {
        final var result = actuatorClient.get()
                .uri(uriBuilder -> uriBuilder
                        .pathSegment("swagger-ui")
                        .build())
                .accept(MediaType.TEXT_HTML)
                .exchange()
                .expectStatus().isFound()
                .expectHeader().location("/actuator/swagger-ui/index.html")
                .expectBody()
                .returnResult()
                .getResponseBody();
        assertThat(result).isNull();
    }

    @Test
    void readinessProbeShouldBeCollectedFromApplicationMainPort() {
        final var result = webTestClient.get()
                .uri(uriBuilder -> uriBuilder
                        .pathSegment("readyz")
                        .build())
                .accept(MediaType.APPLICATION_JSON)
                .exchange()
                .expectStatus().isOk()
                .expectBody(String.class)
                .returnResult()
                .getResponseBody();
        assertThat(result)
                .isEqualTo("{\"status\":\"UP\"}");

        final String metricsResult = actuatorClient.get()
                .uri(uriBuilder -> uriBuilder
                        .path("prometheus")
                        .build())
                .accept(MediaType.valueOf("text/plain"))
                .exchange()
                .expectStatus().isOk()
                .expectBody(String.class)
                .returnResult()
                .getResponseBody();
        assertThat(metricsResult)
                .contains("http_server_requests_seconds_bucket");
    }
}

Full example here.

What code coverage should there be?

Often you use code coverage as a target. you can come across a figure of 80%. Someone calls it acceptable range of 70%-90%So how much should it be?

I really like Robert Martin's thesis – the more, the better. Ideally, of course, you should strive for 100% coverage, but only for selected critical components and applications (general libraries, mission critical and business critical services). Here I would start from what percentiles by availability or latency you look. Is it enough for you p95 or needed p99? Why should code coverage by tests be less?

At the same time, the larger your code base, the more difficult it is to achieve high coverage rates. For a library of 2-3 thousand lines of code, 100% coverage fairly easy to achieveFor a microservice of the same size, getting the coveted 100% is much more difficult.

As a rule of thumb, you can subtract one percent from a hundred for every 1-2 thousand lines of code. For example, if you have a 10 thousand line project, then 90% coverage (100% – 10 * 1%) will be quite a decent indicator. Again, this rule is not very applicable to large and legacy projects.

Conclusion

Learn to test your code automatically. Over the course of several years, this will save you a huge amount of time.

Make testing a habit and a routine. Think of it like brushing your teeth twice a day.

Test a wide range of behaviors in your application, including infrastructure-related ones (e.g. actuator tests).

And then tests will become your faithful friends and helpers.