Java in the clouds
The future is coming. It is already very difficult for us to imagine how people lived without constant access to the Internet, how they met without having phones, how they went to photo salons and then kept rare photos all their lives, and even how to go to work in the office every day – many have already forgotten. I think that very soon deploying applications on physical servers will become a thing of the past. Everyone will go into the clouds.
The trouble is that our beloved Java is not particularly suitable for hosting in the cloud. Thanks to the “write once, work everywhere” policy, we have a complex multi-stage service preparation process that takes up a lot of memory and CPU time, for which we have to pay very heavily to the cloud service.
Fortunately, as I already wrote, the future has come, and we have some technologies that make the life of a service in the cloud cheaper and more enjoyable. Today we’ll look at two of them: Native Image technology with GraalVM and CraC technology.
My name is Sultanov, and I am a team lead (heavy sigh). I try to make applications fast. Sometimes it even works out. And I also have channel, where you can discuss this and other articles. Subscribe, it's interesting.
Why was all this invented?
Is it possible to host a regular Java application on some stock JDK in the cloud? Certainly. To do this, you just need to put the hosted service into a Docker container, run it under Kubernetes, and everything will work great. Well, that is, how wonderful…
The stock JDK loves JIT compilation, which means it spends a lot of CPU (for which you need to pay the cloud provider), besides, it stores heuristics and statistics in memory, the compiled code in memory, and does not start compilation immediately, but only after interpretation sessions, and only when it has accumulated enough statistics. There are several stages of JIT compilation, the classloader also drags everything into memory, and GarbageCollector also needs to clean up after this.
In general, everything is not so great. If we need to deploy one instance of a low-load service, it seems no big deal and especially expensive, but if we need to constantly deploy and extinguish services, balancing the load, it’s already a bit expensive. If our service must immediately provide a quick response, and this is critical for business, then we will face complete failure.
In fact, quite a lot of solutions have been invented, today we will look at two of them. And he will be the first to speak
GraalVM with Native Image technology
According to information on the official website, the technology not only has the highest performance, it is also polyglot (that is, it can work with several languages). Let's try to get to the heart of the solution being used.
How it works
The main feature of Native Image technology is AOT – Ahead of Time compilation. This means something like this: instead of the long process of building an application on the default JDK, GraalVM creates the same Native Image, that is, an application image that is already ready for containerization.
Unlike JIT mode, in which compilation and execution occur simultaneously, in AOT mode the compiler performs all operations at build time, before execution. The basic idea is to move all the “hard work” – the expensive calculations – to the image generation stage so that it can be done once, and then at runtime the generated executables run quickly and are ready from the start because everything is up front calculated and precompiled.
The GraalVM “native-image” utility takes Java bytecode as input and outputs a native executable. This happens in three stages:
Analysis of entry points (usually the main method). GraalVM Native Image determines which Java classes, methods and fields are reachable at runtime, and only those will be included in the native executable. The analysis iteratively processes all transitively reachable code paths until a fixed point is reached and the analysis ends. This applies not only to the application code, but also to the JDK libraries and classes—everything needed to package the application into a standalone binary file.
Initialization at build time. GraalVM Native Image by default initializes the class at runtime to ensure correct behavior. But if the class has no external dependencies or accesses external resources, it will be initialized at build time. This eliminates the need for initialization and runtime checks and improves performance.
Creating heap snapshots. Everything that is initialized is stored in a heap cast, and then simply loaded into memory at the time the application starts, making warm-up unnecessary.
The listed measures to optimize the start of the application really lead to good results. The application starts much faster.
BUT!
There are no miracles, and any technology that has wonderful advantages carries with it a reflection of these advantages – vile disadvantages. And GraalVM has them too.
The presented technology completely changes the approach to application development. How was it before? I wrote it, debugged it, and forgot about it. All further work could be entrusted to the long, but very stable and repeatedly tested compilation process, sometimes rearranging flags in the JVM.
Now you need to think in advance, what will happen when creating the image? How will the application behave under AOT? Can a smart machine cut out something needed from an image?
This problem is generally recognized by the authors of the Native image technology themselves. For example, in order for reflection to work in your application, you need to maintain a manually (which is unpleasant) or automatically (which is dangerous) configuration file that allows reflection of some parts of the program. It actually looks like a crutch.
GraalVM also does not use the most advanced garbage collector (G1), has poorly debugged code, the IDE does not highlight errors related to Native Image, and the technology is Linux-specific. Over time, this will of course be fixed, but we live and develop applications at the moment. Now it is completely unclear what difficulties will be encountered in production using Native image. But, as they say, who doesn’t take risks…
CRaC – Coordinated Restore at Checkpoint
One of Native image’s competitors in speeding up and reducing the cost of launching applications in the cloud is CRaC technology. In general, the whole essence is stated in the name of the technology. CRaC is about creating application images and restoring from them.
How it works?
In the Linux kernel (yes, CRaC is also Linux-specific), another technology has existed for a very long time, namely CRIU – Coordinated Restore in Userspace. This is what CRaC uses internally. CRUI allows you to freeze any process in the form of a set of some files, and see the full contents of memory, all the states of the threads that were working at that moment, that is, it simply saves a snapshot of the process state. And this snapshot (in the form of a set of files) can be saved somewhere and then launched, after some time or even on another physical machine, and it will seem to not notice anything, and will start working the same way as it worked before, with the same place.
Actually, CRaC is an adaptation of CRIU technology for Java. Seems simple and should work great. As measurements show, the startup time from the image is two orders of magnitude less than the time of traditional application startup on the stock JVM.
BUT!
As always, there are problems. Our application does not work on its own; it always has connections, which are also saved in the application snapshot. To prevent this from happening, a certain procedure called coordination is needed. At the most primitive level, this is done like this:
That is, we must manually handle closing all resources before creating the image, and then opening these resources again. If this is not done, the system will simply refuse to create the image.
A consequence of creating a system image is security issues. The image itself cannot be shown to anyone, as all the ins and outs are there. Compromising the image would be a security disaster.
Another problem will be the size of the image, which will essentially be similar to the size of the heated JVM application.
Conclusion
At the moment, both technologies demonstrate potential, but at the same time they also show problems that are commonly called childish. The developers are solving them, but when they decide, and when there will be sufficient guarantees to use Native image and CRaC in production, and not turn gray, is still unclear.
