I don't like the Codeium neural network

Many companies are now trying to integrate AI assistants into their development environments in hopes of making programmers' jobs easier and more efficient. Earlier this year, I participated in a pilot test of a free Codeium. The neural network was used out of the box (without additional training) and was tested during development in C# in Visual Studio 2022 (production code and tests). In general, I did not like the performance of Codeium and here's why.

Understanding the context

It seems that the neural network only looks in the context of the current file. Incorrectly advises to use fields and methods (public!) from other classes, simply generates random names. Cannot work with third-party libraries.

Generates code better if the code contains comments. I tried adding comments in some places specifically to check the behavior of the neural network. And yes, Codeium starts to react faster in such cases and recommend more expected code. This is inconvenient for me, because I do not leave the number of comments in the code that Codeium requires. Another feature that was noticed is that for completely new code it generates suggestions better, while when refining the current one – worse.

When declaring a new test method, it may advise calling the method. I encountered this behavior several times during the month. In my experience, development environment assistants never suggest calling a function instead of declaring it in this place:

[Test]
public void Test_1()
{
    ...
}

// Ставлю курсор здесь и получаю рекомендацию от Codeium
Test_2();

[Test]
public void Test_2()
{
    ...
}

Quality of generated code

As for syntactic sugar: recommends something from older language standards. After accepting the generated code, it has to be edited, which takes time.

Speed ​​of work

Offers a code, but when accepting it, it may add another line that was not in the preview. It seems like there is some kind of time delay.

Often unable to offer any options at all.

It is annoying that after installing the Codeium extension, Visual Studio experienced some slowdowns. After removing Codeium, the development environment works as usual again.

Further training of the neural network

The creators of Codeium promise to improve the quality of the generated code by further training the neural network. There are two options, and there are questions about each of them:

• Training on the product codebase

  Large projects contain code of completely different quality and, of course, a lot of legacy code. The question is, what to teach on? Which code is more suitable? Will we consider five-year-old code acceptable?

• The generated code is accepted, which means it is correct.

  When working with a neural network, programmers often adhere to the position: you accept the generated code, even if it is not entirely correct, and then edit it to the desired form. But if the neural network is capable of learning, then accepting the recommended code is interpreted by the neural network as the correct option. In this case, the neural network will never understand that the code is incorrect.

There is one thing that I really liked

The only thing you save time on when using Codeium is that the neural network autocompletes comments (in Russian and English) very well. And that's all for me. But I don't think it's worth installing Codeium for that 🙂 I remember using ReSharper at my previous job, which also did a great job with this.

After using Codeium for a month, I discovered many disadvantages, and they outweigh the advantages. Out of the box, the Codeium neural network works weakly. In my opinion, there are few arguments for implementing Codeium as an everyday assistant.

P.S. Neural networks are great at working with images, and they successfully help in medicine for diagnostics. This year, videos created with the help of neural networks are especially pleasing. It is enough to recall the viral video during the 2024 presidential election campaign.