Creating and testing annotation processors (with code generation) for Kotlin


In Kotlin (or Java) development, annotation marking (for example, for database table models, network requests, or dependency injection) and the inclusion of annotation processors that can also generate code that is accessible from the main project are often used to create classes from a top-level description. Running annotation processors is done inside gradle (for Java projects via annotationProcessor, for Kotlin – kapt) and built in as a dependency for project build purposes. And of course, like any other code, an annotation processor needs to be able to develop tests. In this article, we will cover the basics of using code generation (using kapt) and developing tests for generated code generators. In the second part of the article, we will talk about the development of processors based on Kotlin Symbol Processing (KSP) and the creation of tests for them.

Let’s start with the classic code generation mechanism kapt (Kotlin Annotation Processing Tool). kapt is built into gradle (as a plugin) or into maven (by adding <goals><goal>kapt</goal></goals> to description execution in the project configuration). In general, the project configuration with kapt can be as follows:

plugins {
    kotlin("jvm") version "1.8.20"
    kotlin("kapt") version "1.8.20"
    application
}

group = "tech.dzolotov"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

kotlin {
    jvmToolchain(17)
}

application {
    mainClass.set("MainKt")
}

After connecting code generation to kapt, it becomes possible to connect processors through the kapt command, for example, Autovalue code generation is connected to create immutable classes (in fact, in Kotlin they can be implemented through data classes and AutoValue solves the same task for Java and is in many ways similar to Lombok, but works otherwise).

dependencies {
    implementation("com.google.auto.value:auto-value-annotations:1.10.1")
    kapt("com.google.auto.value:auto-value:1.10.1")
}

The kapt processor works similarly to the Java annotation processor, but it first converts the Kotlin source code to Java code and then passes it to the generator. This generally reduces the speed of code generation (even compared to a Java project), and to solve this problem, an alternative KSP mechanism was created, which we will discuss later. In principle, code generation can create code not only in Java, but also in Kotlin or any other language, but many generators used with kapt are designed natively for Java (for example, Room, Hilt, etc.).

Let’s add a simple class to describe users with automatic identifier detection:

import com.google.auto.value.AutoValue

@AutoValue
abstract class UserInfo {
    abstract fun getId(): Int
    abstract fun getLogin(): String
    abstract fun getPassword(): String

    companion object {
        var id = 0
        fun create(login:String, password:String):UserInfo {
            id++
            return AutoValue_UserInfo(id, login, password)
        }
    }
}

To perform code generation, run the gradle task (:

./gradlew kaptKotlin

The generated code is mostly located in the build directory (build/generated/source/kapt/main) and is Java source code (in addition to creating get methods, it also overrides equals, hashCode And toString). There is no need to import it separately, since it is placed in the same package as the original annotated class. The generated class will be annotated with the annotation @Generated with an indication of the processor class that created this class:

@Generated("com.google.auto.value.processor.AutoValueProcessor")
final class AutoValue_UserInfo extends UserInfo {
  //определения полей
  //get-функции
  //toString, equals, hashCode
}

Now let’s make a code example for using the generated class:

fun main() {
    val users = mutableListOf<UserInfo>()
    users.add(UserInfo.create("user1", "password1"))
    users.add(UserInfo.create("user2", "password2"))
    println(users)
}

The result will be a string representation of the list:

[UserInfo{id=1, login=user1, password=password1}, UserInfo{id=2, login=user2, password=password2}]

Now let’s deal with creating our own code generator. We will use as a basis sample of three modules (a module with an application that will use the annotation processor, a module with an annotation, and a processor module). Let’s define an annotation to use in the code generator:

@Retention(AnnotationRetention.SOURCE)
annotation class SampleAnnotation

And we implement the processor itself, which will be defined in the process method in the extension class from javax.annotation.processing.AbstractProcessor:

@AutoService(Processor::class)
@SupportedSourceVersion(SourceVersion.RELEASE_17)
@SupportedAnnotationTypes("kaptexample.annotation.SampleAnnotation")
class SampleAnnotationProcessor : AbstractProcessor() {
    override fun process(annotations: MutableSet<out TypeElement>, roundEnv: RoundEnvironment): Boolean {
        roundEnv.getElementsAnnotatedWith(SampleAnnotation::class.java).forEach {
            processingEnv.messager.printMessage(Diagnostic.Kind.WARNING, "${it.simpleName} is processed.")
        }
        return true
    }
}

Here the auto-service library is used to automatically register the class as a code generation processor (included in build.gradle.kts):

dependencies {
//...
    implementation("com.google.auto.service:auto-service:1.0.1")
    kapt("com.google.auto.service:auto-service:1.0.1")
}

The process method will be called when the annotations listed in @SupportedAnnotationTypes and can have access to source code definitions through the implementation RoundEnvironment (receives in roundEnv). Also inside AbstractProcessor have access to processingEnvthrough which you can receive arguments for kapt (via options), create files (via the field filer) and output messages to the IDE and the gradle console (the type from the Diagnostics.Kind enumeration is specified for the message: ERROR – on error WARNING displayed as an informational message, OTHER – for any other type of message that does not interrupt the execution of code generation). Through roundEnv, you can get information about annotated definitions (can be before a package, interface / class, function / method, or variable definition), each definition is represented by an implementation of the Element interface and allows you to get meta information about the definition:

  • simpleName – name (without package)

  • kind – element type (defined in ElementKind)

  • getAnnotation(type) – getting the annotation object (together with arguments, if defined)

  • modifiers – definition modifiers (for example, private or static)

  • enclosingElement – gives access to a top-level element (for example, a class definition for an annotated method)

  • enclosedElements – returns a list of nested elements (for example, property and method definitions for an annotated class)

Let’s define a simple class (the @JvmField annotation is used here to prevent automatic generation of get methods).

@SampleAnnotation
class SampleClass {
    @JvmField
    val x: Int = 0
    @JvmField
    val y: Int = 0
}

and create a processor that will detect and display all found class properties:

@AutoService(Processor::class)
@SupportedSourceVersion(SourceVersion.RELEASE_17)
@SupportedAnnotationTypes("kaptexample.annotation.SampleAnnotation")
class SampleAnnotationProcessor : AbstractProcessor() {
    override fun process(annotations: MutableSet<out TypeElement>, roundEnv: RoundEnvironment): Boolean {
        roundEnv.getElementsAnnotatedWith(SampleAnnotation::class.java).forEach { outer ->
            outer.enclosedElements.forEach {inner ->
                if (inner.kind== ElementKind.FIELD) {
                    processingEnv.messager.printMessage(
                        Diagnostic.Kind.WARNING,
                        "Field ${inner.simpleName}, modifier: ${inner.modifiers}"
                    )
                }
            }
        }
        return true
    }
}

Now let’s add code generation, for this we get a Filer object and through it we can create a bytecode (createClassFile), resource (createResource) or generate a new source file (createSourceFile). Next, the created file can be accessed via writer and write the generated source text there (after the work is completed, the generated file will be checked for correct syntax). For example, we want to add an id field with auto-increment, for this we first prepare a source code template (in Java, but also in Kotlin):

public class GeneratedSampleClass {
  GeneratedSampleClass(<список полей>) {
    //заполнение полей по значениям из конструктора
  }
  static int id = 0;
  int getId() {
    id++;
    return id;
  }
  //здесь подставляем определение полей из исходного класса
}

using template and information from discovered objects (package name is retrieved from enclosingElement for annotated class, field definition name and signatures from enclosedElements from class)

@AutoService(Processor::class)
@SupportedSourceVersion(SourceVersion.RELEASE_17)
@SupportedAnnotationTypes("kaptexample.annotation.SampleAnnotation")
class SampleAnnotationProcessor : AbstractProcessor() {
    override fun process(annotations: MutableSet<out TypeElement>, roundEnv: RoundEnvironment): Boolean {
        roundEnv.getElementsAnnotatedWith(SampleAnnotation::class.java).forEach { outer ->
            val fields = mutableListOf<Element>()

            var pkgName:String? = null
            val pkg = outer.enclosingElement
            if (pkg.kind==ElementKind.PACKAGE && pkg.toString()!="unnamed package") {
                pkgName = pkg.toString()
            }
            processingEnv.messager.printMessage(Diagnostic.Kind.WARNING, "Package is $pkgName")

            outer.enclosedElements.forEach { inner ->
                if (inner.kind == ElementKind.FIELD) {
                    fields.add(inner)
                    processingEnv.messager.printMessage(
                        Diagnostic.Kind.WARNING,
                        "Field ${inner.simpleName}, modifier: ${inner.modifiers}"
                    )
                }
            }
            processingEnv.messager.printMessage(Diagnostic.Kind.WARNING, processingEnv.options.toString())
            val className = "Generated${outer.simpleName}"
            val classFile = processingEnv.filer.createSourceFile(className)
            classFile.openWriter().use {
                val varFields = fields.map { "${it.asType()} ${it.simpleName}" }
                var initFields = fields.map { "this.${it.simpleName} = ${it.simpleName};" }
                val definitions = mutableListOf<String>()
                fields.map { field ->
                    //добавляем модификатор для доступа к полю (и исключаем дублирование)
                    val accessModifiers = listOf("public", "private", "protected")
                    definitions.add("public ${field.modifiers.filter { !accessModifiers.contains(it.toString())}.joinToString(" ")} ${field.asType()} ${field.simpleName};")
                }
                it.write(
                    """
                    ${if (pkgName!=null) "package $pkgName;" else ""} 
                    public class $className {
                    
                      public $className(${varFields.joinToString(",")}) {
                        ${initFields.joinToString("\n")}
                      }
                    
                      static int id = 0;
                      public int getId() {
                        id++;
                        return id;
                      }
                      //здесь подставляем определение полей из исходного класса
                      ${definitions.joinToString("\n")}
                    }
                """.trimIndent()
                )
            }
        }
        return true
    }
}

Here, access modifiers are additionally replaced with public (so that the test can later read the fields, you can alternatively add the generation of get methods). It is also important that the class itself and the constructor are public, otherwise an error will occur at the stage of creating an object through reflection. Similarly, you can generate any data structures and code fragments.

Libraries can also be used to generate code, for example JavaPoet makes it possible to represent the code in the form of a tree of objects and generate formatted code in the Java language.

Now let’s move on to testing the developed code generator. To do this, we will connect the kotlin-compile-testing library and add our project for

dependencies {
  testImplementation("com.github.tschuchortdev:kotlin-compile-testing:1.5.0")
}

The library allows you to programmatically compile a given code fragment in Java or Kotlin (you can add annotation processors, including KSP). It is important to add the annotation definition file to the compilation, because when building, the library does not know about the existence of gradle projects and works directly with the code fragment.

Let’s start with a simple test of a small class without the use of code generation:

     @Test
    fun testSimpleCode() {
        val result = KotlinCompilation().apply {
            sources = listOf(SourceFile.kotlin("MySimpleTest.kt", """
                class Calculator {
                    fun sum(a:Int, b:Int) = a+b
                }
            """.trimIndent()))
        }.compile()
        assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode)
    }

The resulting result object contains information about the generated files in the field generatedFiles (in this case, only the source text, bytecode and META-INF), you can also find out the compilation result (exitCode), get a list of files created by annotation processors (sourcesGeneratedByAnnotationProcessor), as well as access the class loader to reflect on the created class and create its instances through constructors and newInstance. Let’s add sum method signature tests and check the functionality of the created class:

    fun testSimpleCode() {
        val result = KotlinCompilation().apply {
            sources = listOf(SourceFile.kotlin("MySimpleTest.kt", """
                class Calculator {
                    fun sum(a:Int, b:Int) = a+b
                }
            """.trimIndent()))
        }.compile()
        assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode)
        val calculatorDescription = result.classLoader.loadClass("Calculator")
        assertDoesNotThrow("Sum method is defined") { calculatorDescription.getDeclaredMethod("sum", Int::class.java, Int::class.java) }
        val calculatorInstance = calculatorDescription.constructors.first().newInstance()
        assertEquals(8, calculatorDescription.getDeclaredMethod("sum", Int::class.java, Int::class.java).invoke(calculatorInstance, 3, 5))
    }

It is important to remember here that the creation of objects, the execution of methods and access to properties take into account accessibility modifiers and, since the test code is now in a different package, you need to make sure that the corresponding modifiers are public.

Now let’s move on to testing our code generator. To add annotation processors, the KotlinCompilation (or JavaCompilation) class object has a list in the annotationProcessors property:

    @Test
    fun testCodegen() {
        val result = KotlinCompilation().apply {
            annotationProcessors = listOf(SampleAnnotationProcessor())
            val source = SourceFile.kotlin("MyTestClass.kt", """
                import kaptexample.annotation.SampleAnnotation                

                @SampleAnnotation
                class MyTestClass {
                    val x:Int = 1
                    val y:Double = 0.0
                }
            """.trimIndent())
            //подключаем аннотацию
            val ann = SourceFile.fromPath(File("../kapt-example-core/src/main/kotlin/kaptexample/annotation/Sample.kt"))
            this.sources = listOf(source, ann)
        }.compile()
        //проверим успешность компиляции
        assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode)
    }

Here is the source text with the annotation definition (in kapt-example-core) is compiled along with our snippet so that it works correctly import and applying the annotation. The further test is performed similarly to the previous example:

    fun testCodegen() {
        //...компиляция кода (из предыдущего примера)
        //--------------------------------------------
        //можно проверить отображенные сообщения через result.messages
        //используем рефлексию для проверки результата
        val rc = result.classLoader.loadClass("GeneratedMyTestClass")
        assertDoesNotThrow("getId is defined") { rc.getDeclaredMethod("getId") }
        assertEquals(3, rc.declaredFields.size, "Valid fields")
        assertContentEquals(rc.declaredFields.map { it.name }.sorted(), listOf("x", "y", "id").sorted())
        assertEquals(1, rc.declaredConstructors.size)
        assertEquals(2, rc.declaredConstructors.first().parameters.size)
        //создаем экземпляр объекта через конструктор
        val instance = rc.constructors.first().newInstance(2, 3.0)
        //здесь мы не имеем доступа к определению объекта, поэтому вызываем через invoke от метода
        assertEquals(1, rc.getMethod("getId").invoke(instance))
        assertEquals(2, rc.getField("x").get(instance))
        assertEquals(3.0, rc.getField("y").get(instance))
        //проверим создание второго экземпляра и корректное заполнение id
        val instance2 = rc.constructors.first().newInstance(5, 8.0)
        assertEquals(2, rc.getMethod("getId").invoke(instance2))
    }

The source code for the project can be found in the repository https://github.com/dzolotov/kapt-template (codegen-test branch).

We have covered the basic issues of developing annotation processors with code generation capability for Java or Kotlin projects and how to test their correctness. In the second part of the article, we will explore a new approach to generating Kotlin code using Kotlin Symbol Processing (KSP) and, of course, learn how to develop tests for KSP processors.

In conclusion, I invite everyone to free webinar within which we will learn how to check the readiness of a mobile application for use by people with disabilities. Also, the readiness to automatically check compliance with the requirements of visual contrast, adaptation of the layout to an enlarged font, the presence of semantic markup for auxiliary tools for Android applications (XML and Compose) and iOS (Flutter and KMM). We will learn how to use automated checks tools and create custom validators to implement complex visual checks.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *