Protobuf vs Reflection

Have you ever wondered how grpc fast. Yes, on the Internet, he has no equal. If you are sending small messages that need to be delivered quickly, then grpc simply cannot be found better. But how good is he? Will it be able, for example, to simply compare with native calls?

Let’s try to compare this, but since in ordinary life we ​​may not need this, we will add one more condition – we compare it as the best way to interact with jni library.

Formulation of the problem

Since ancient times, reflection in jvm is considered to be a very long and costly process, every self-respecting developer could kick June at any review – what kind of nightmare code he uses through reflection. Now times are different, now June has become smarter, but the need to use reflection has not disappeared.

For example, if you are writing ndk applications, then with almost 100% probability you will need to use this approach to simply map data from jvm into classic C++ classes or structures. And yes, this mapping is not for the faint of heart.

JNIEXPORT jint JNICALL
Java_com_example_engine_JniEngine_cmd(JNIEnv *env, jobject, jobject jCmd) {
    struct some_cmd cmd{};

    jfieldID idField = env->GetFieldID(env->FindClass("com/example/model/SomeCmd"), "id", "I");
    jfieldID nameField = env->GetFieldID(env->FindClass("com/example/model/SomeCmd"), "name", "Ljava/lang/String;");

    cmd.id = env->GetIntField(jCmd, idField);
    
    jstring jName = (jstring) env->GetObjectField(jCmd, nameField);
    const char *name = jName != NULL ? env->GetStringUTFChars(jName, NULL) : NULL;
    cmd.name = std::string(name ?: "");
    if (name != NULL) env->ReleaseStringUTFChars(jName, name);
}

This is approximately how you need to map data from jvm in the native library ndk. And how many problems happen at once if, for example, someone decides to move the model SomeCmd from one package to another. Well, or rename the field or method.

Each such mapping can use reflection one or more times.

jfieldID idField = env->GetFieldID(env->FindClass("com/example/model/SomeCmd"), "id", "I");
jfieldID nameField = env->GetFieldID(env->FindClass("com/example/model/SomeCmd"), "name", "Ljava/lang/String;");

This is a rather expensive process, especially if there are many fields and methods that we are looking for. It seems that simple serialization of models into some binary format will speed up this process, and will also save us from the need to clean up resources.

Just Proto

Having chosen the path of war, we plan to test everything on our computer. And since there are not many monstrous plugins that support both cmake and java, the author can only mention AGP, and it is not suitable for us due to the specificity of the platform, then we choose our own path. We build a project from several plugins for cmake, java, and protobuf.

plugins {
    application
    id("com.github.gmazzo.buildconfig") version ("3.1.0")
    id("io.github.tomtzook.gradle-cmake") version ("1.2.2")
    id("com.google.protobuf") version ("0.9.3")
}

We use the classic protobuf implementation without grpc And kroto plugins. The test itself does nothing, just sends a simple model to the native library and back. The model was made nested and from different types of data to get closer to real use.

syntax = "proto3";

option java_package = "com.github.klee0kai.proto";

message SomeCmdModel {
  int32 id = 1;
  int64 count = 2;
  float value = 3;
  double valueD = 4;
  string name = 5;
  repeated MetaModel meta = 6;
}

For a cmake project, you additionally need to install the protoc libraries, for example for ubuntu.

apt-get install libprotobuf-dev protobuf-compiler

We generate C++ models from proto files and add the library to CMakeLists.txt.

find_package(Protobuf REQUIRED)
include_directories(${Protobuf_INCLUDE_DIRS})
include_directories(${CMAKE_CURRENT_BINARY_DIR})
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS ./../proto/jni.proto)

target_link_libraries(myapplication ${Protobuf_LIBRARIES})

target_include_directories(
        myapplication
        PUBLIC
        ${JNI_INCLUDE_DIRS}
        ${PROTO_HDRS}

        ./../../../build/generated/sources/headers/java/main
)

The header files themselves are generated by the protobuf plugin in gradle, and working with protobuf in C++ will already work due to the library found in cmake. Check the versions of the libraries used both here and there, since the protobuf library found in cmake may not digest the generated header files or may not support various options, such as lite.

Since our project uses several independent plugins that are in no way related to each other, we slightly adjust the sequence of tasks in gradle. We will build cmake after java – this is necessary so that jni header files for C++ are generated, and also header files for proto models are generated, which in our case are generated via gradle.

tasks.clean.dependsOn(tasks.cmakeClean)
tasks.classes.dependsOn(tasks.cmakeBuild)
tasks.cmakeBuild.dependsOn(tasks.compileJava)

The project itself, by the way, is written in Java. Jni integration for kotlin looks more complicated. So, for example, many kotlin types in jni are not accessible by their class names:kotlin.Int represented either through a primitive type intor through its object representation java.lang.Integer. Kotlin does not generate header files for jni, which is supported by default in java. In addition, kotlin can distort final names of fields and methods.

Having collected all the plugins together, we get the following sequence of project assembly.

:term:run
\--- :term:classes
     +--- :term:cmakeBuild
     |    +--- :term:compileJava
     |    |    +--- :term:generateBuildConfig
     |    |    \--- :term:generateProto
     |    |         +--- :term:extractIncludeProto
     |    |         \--- :term:extractProto
     |    \--- :term:myapplication_linux-amd64_runGeneratorUnix_Makefiles
     |         \--- :term:cmakemyapplication_linux-amd64
     +--- :term:compileJava
     |    +--- ***
     \--- :term:processResources
          \--- :term:extractProto
               \--- ***

All that remains is to assemble and run the test.

./gradlew run

Dry residue

After running the test we get the result

> Task :term:run
test jni reflection test...........
Test time 1.272
test on indexed jni reflection...........
Test time 0.594
test proto serialize...........
Test time 1.034

Protobuf turned out to be faster than reflection, but is not the best solution for working with jni. The most optimal solution was to use reflection with preliminary indexing of classes and methods – which took about 0.594 seconds in total. work for 100k operations of copying and sending models to the jni library and back.

Since the author did not measure the speed of native java work, we will assume that indexed reflection is its most approximate version. Losses when using protobuf will be x2 from native work, and when reflecting – about x2.2. For more information about the work, see github.