How without money and knowledge in AI repeated Copilot'a

I must say right away that this is not a clickbait, but I financially (like most likely you) could not bring this idea to the end.

To begin with, let’s look at Copilot from all sides!

It was developed jointly Github(dataset from them) and Open AI(from them Codex)
It was abruptly closed in Russia due to sanctions
He continues the code trying to understand your idea.

Agree, very interesting!

Now let’s see what we have, and spend as little money as possible…

It is not problematic to get the dataset of the code, it’s been out in the open for a long time..

Codex is expensive, so I decided to find an Open Source competitor, and to my surprise, I found it!

CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.

——-

CodeGen is an open source model for program synthesis. Trained on TPU-v4. Competes with OpenAI Codex.

The developers of a certain CodeGen claim that their product competes with Codex, but believing in a word is one thing, and testing it in practice is another. So after 5 minutes I built Google Colab, and started checking.

Link to Colab

https://colab.research.google.com/drive/1ouZ2uyJC0ZNSYoThCBuRl35BBmYQZiLK#scrollTo=UUq-rqNGOa0L

By the way, I noticed an interesting point when I studied the documentation:

There are several variations of trained models that are divided by size and trained data, which made me very happy. Why? And let’s get back to this!

For tests, I decided to take a medium-sized model with all the options, this model weighs almost 6GB in the end:

And here arises (at least for me) a few questions:

Copilot generates a response in a second, will CodeGen be just as good?
What languages does CodeGen support?

With the first question, everything is not very clear, I write after all with minimal knowledge in this field, but the Colab machine with 16GB of video memory generates on the model Salesforce/codegen-2B-mono a response with a length of up to 24 characters in almost a second.

And I found the answer to the second on the model’s website, C, C++, Go, Java, JavaScript, and Python.

Interesting fact

By the way, the most powerful model weighs 32GB

We continue, now it’s time to test what we have:

Several conclusions can be drawn from the picture above, the average model already easily understands what they want from it, but it generates a response in 1.44 seconds…

Yes, it’s sad, that’s the catch, financially it’s expensive to maintain such a project, because the prices for a server with video cards start at 11 thousand, and those won’t work for us to file a full-fledged Copilot, but for yourself you can raise it on your PC or Colab easily.

In fact, I can’t give a big continuation of the project, but I showed that everything can be replaced with alternatives if you make an effort, good luck!