Open Source Tools for GPU Computing

GPU computing can be useful for many developers because it allows them to improve the performance of their code. The technology is available, but to speed up code execution or create colorful visualizations, you can’t simply move calculations from the CPU to the GPU — it requires special compilers and libraries.

Today we have collected several interesting solutions that will be useful in a variety of tasks: from accelerating high-load calculations to generating graphics.

ILGP

ILGP — is a compiler for implementing GPU computing in .NET languages, introduced in 2021. It supports just-in-time technology, which translates code into machine language during program execution. ILGPU is distributed under the open source NCSA license. It allows you to use, modify, and distribute the software while maintaining the copyright notice.

ILGPU is written in C# and combines the capabilities of C++ AMP and CUDA. The solution provides high dispatch speed, and type-safe delegates allow to avoid boxing, i.e. converting the language's dimensional data types from value to reference. ILGPU also allows to adapt the code to the target architecture, be it x86 or x64, without the need to make manual changes.

At the same time, the compiler supports work not only with static, but also with dynamic shared memory. The latest versions of ILGPU allow it to be allocated for each core separately.

class ...
{
    static void SharedMemKernel(ArrayView<int> data)
    {
        var staticMemory = SharedMemory.Allocate<int>>(1024);
        var dynamicMemory = SharedMemory.GetDynamic<int>>();
        ...
        var dynamicMemory2 = SharedMemory.GetDynamic<double>>();
        ...
    }
    static void ...(...)
    {
        using var context = Context.CreateDefault();
        using var accl = context.CreateCudaAccelerator(0);
        var config = SharedMemoryConfig.RequestDynamic<byte>(<GroupSize> * sizeof(int));
        var config2 = SharedMemoryConfig.RequestDynamic<int>(<GroupSize>);

        ...
        kernel((<UserGridDim>, <UserGroupDim>, config), buffer.View);
        ...
    }
}

The compiler allows you to use standard .NET debuggers to find and fix errors in your code and supports single-threaded execution of programs.

Other ILGPU capabilities include atomic operations and low-level mechanisms such as warp mixing functions. Support for high-performance math functions further speeds up calculations, especially when working with 32-bit arithmetic. Interestingly, the compiler was used to simulate the game of Life with a refresh rate of 300+ FPS.

The tool also includes a library ILGPU.Algorithms. It offers a set of high-level auxiliary algorithms, such as sorting and prefix sum computation, that are supported on all types of accelerators.

For more information, please see repositories on GitHub And Discord communitywhose participants are ready to share their experiences.

ComputeSharp

This – library .NET for working with graphics, which is distributed under the MIT license. It was developed by Microsoft several years ago, and is still used in the company's products: Microsoft Store and Paint.NET.

ComputeSharp APIs allow you to access GPU devices, move data between GPU buffers and RAM, and write shaders in C#.

ComputeSharp supports multidimensional thread group dispatching and allows you to use a variety of data types, including .NET vector types and HLSL for complex calculations.

For example, here shader examplewhich applies the multivariable logistic function (softmax function) to all elements in a given buffer. It is typically used in classification problems in machine learning.

[ThreadGroupSize(DefaultThreadGroupSizes.X)]
[GeneratedComputeShaderDescriptor].
public readonly partial struct SoftmaxActivation(
    ReadWriteBuffer<float> buffer,
    float k) : IComputeShader
{
    public void Execute()
    {
        float exp = Hlsl.Exp(k * buffer[ThreadIds.X]);
        float log = Hlsl.Log(1 + exp);

        buffer[ThreadIds.X] = log / k;
    }
}

It is worth considering that working on DX12 limits the applicability of the library. At the very least, this fact makes ComputeSharp less universal compared to its analogues, for example, ILGPU, ShaderGen.

gpu.cpp

This is open library for GPU computing in C++. It was introduced in July 2024 by Answer AI, a research lab founded by the former CEO of Kaggle. The library is distributed under the Apache 2.0 license.

The library developers note that solutions for computing on personal devices are mainly intended for game engines and ML compilers, but do not take into account the interests of programmers from other areas. The project has many applications, including autonomous graphics rendering and building simulations of processes occurring in the real world (various examples are in the repository).

Under the hood, gpu.cpp is a Dawn library for providing basic WebGPU functionality. Dawn — is a cross-platform, open-source implementation of the WebGPU standard that provides the core tools for developing WebGPU applications, including C/C++ headers and support for D3D12, Metal, Vulkan, and OpenGL. The library has a built-in shader language compiler called Tint.

Stardust

This is a library for rendering using GPU (WebGL). The goal of the project is to simplify and speed up the creation of visualizations without the distraction of managing shaders and API buffers. The library is primarily intended for web developers, data scientists, and designers who need to generate interactive 2D and 3D graphics.

Among the key features of Stardust is real-time rendering — the library processes tens of thousands of graphic elements (called “markers” in the project). The authors provide a simple example on the project's website — code for rendering several geometric figures.

<script type="text/javascript">
    var canvas = document.getElementById("main-canvas");
    var width = 960;
    var height = 500;
    var platform = Stardust.platform("webgl-2d", canvas, width, height);
    var data = [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ];
    var circleSpec = Stardust.mark.circle();
    var circles = Stardust.mark.create(circleSpec, platform);
    circles.attr("center", (d) => [ d * 80, 250 ]);
    circles.attr("radius", (d) => d * 3);
    circles.attr("color", [ 0, 0, 0, 1 ]);
    circles.data(data);
    circles.render();
</script>

The portal also provides complex examples: construction graphs using glyphs and visualizations using three.js. Also in project documentation you can find training materials, and in sandbox try them out in practice.

When it comes to availability of computing resources, virtual infrastructure with GPUs for machine learning, analytics, rendering, and 3D modeling is often mentioned. However, projects are different – virtual desktops with graphics resources (VDI GPUs) effectively cope with 3D rendering and design tasks on regular PCs without the need to purchase expensive graphics stations.

Large-scale projects may require the deployment of a virtual IT infrastructure to solve complex problems. Currently, available cloud services can be used at a discount for six months. You can find out more about this on the website.