Developed by Gamio AI. Through pain and bugs

Gamio AI – a text adventure in Russian based on artificial intelligence.

No matter how pathetic it may sound, I managed to create a working version of this ambitious idea.

Main interface
Main interface

The beginning of all beginnings

I can’t call myself a professional developer, but I had a lot of experience in the field of machine learning.

I wanted to create a product based on neural networks that would be understandable and accessible to everyone without delving into this complex topic.

I didn’t come up with anything better than how to create a text quest using artificial intelligence. This format is already popular abroad.

And so the first version appeared. Gamio AI.

Earliest version of Gamio
Earliest version of Gamio

In the very first version, I resorted to the resources of Ai21 Labs.

At that time, they had two GPT-like models: one had 7 billion parameters, the other 178 billion parameters. They provided the opportunity to train the neural network for free.

I collected CYOA stories and sent them. After a few weeks, I got two models with 7 billion parameters (the second one was based on the previous one). All of them understood only English (I had to use a translator).

Two trained models.
Two trained models.

Two major problems emerged from this.

  • Terrible, simply terrifying quality of the resulting model. (Who is to blame for this is not clear)

  • A huge price to maintain such a model. If every day 100 people play Gamio, then I would need 70,000 rubles / month to keep the project working.

With this attitude, the project worked for two days.

Sample early AI
Sample early AI

Project revival

After several months of painstakingly studying all the available means of implementation, I decided to bring the project to mind no matter what.

Site development

First of all, I started designing the service itself, because I was confident in the quality of the planned artificial intelligence.

I was comfortable with the Python language, so using an open framework Django – was the right decision.

Gamio homepage
Gamio homepage

The site was decided to be designed in a simple, dark style. There were no problems with the creation of the design.

To implement all my creative ideas, I used WebFlow.

captcha

I didn’t want to have bots on my site. Using standard reCAPTCHA or hCAPTCHA didn’t work for me. They can be easily bypassed.

So I decided to invent a bicycle called fiCAPTCHA. This is a standalone Python module that generates an image in which you need to find a specific object.

An example of a generated captcha
An example of a generated captcha

Maybe it’s trite, but the bot definitely can’t get through it.

Here is the code for determining the coordinates of the click and further verification. True coordinates are stored on the server side.

function click(event) {
    var xCoordinate = event.offsetX;
    var yCoordinate = event.offsetY;
    $.ajax({
        type: 'POST',
        url: '/app/captcha/c',
        data: {...},
        success: (function(data){
            ...
        })
    });
}

Censorship

There is a community in Gamio Ai that needs to be regulated.

In this task, it was decided to use artificial intelligence. To be more precise, the BERT model, which classifies the text into two categories: 0 – allow; 1 – do not allow.

Each world and comment has an additional field in its database.

class ...(models.Model):
    ...
    is_reviewed = models.BooleanField(default=False)

During neural network validation, the entry is either deleted or the field is_reviewed takes on the value True

As practice has shown, this decision was correct. AI doesn’t make mistakes.

Artificial intelligence

90% of the success of Gamio Ai depends on the quality of the neural engine that serves the customers.

For the task of generating text, the GPT-2 architecture, which is in the public domain, is excellent.

For the first time, I decided not to train AI from scratch, but to use ready-made solutions. So, I settled on the Russian-language model enGPT3largetrained by Sber.

Fine tuning

The “vanilla” model did not understand what was required of her. Therefore, in this situation, I resorted to fine tuning.

Graph of losses during training
Graph of losses during training

The amount of training data was 45 MB. It’s not enough, but enough.

> Ты [действие]
[результат]

> Ты [действие]
[результат]

The neural network scanned this file 10 times. The result made me happy.

At the moment, this particular model is used in Gamio.

Sample Game
Sample Game

For convenience, the player is given full control over the story.

You can edit any piece of text;

Editing a story
Editing a story

Supplement text through a neural network;

AI addition
AI addition

And the ability to regenerate the last action;

What’s next?

This is just the beginning…

Further, it is planned to train a new Russian-language neural network from scratch, which will have 6 billion parameters. Training should start by the end of June 2022.

If you are interested in Gamio Ai and want to follow its development:


Gamio AI- https://gamio.ru

Telegram – https://t.me/gamio_n

In contact with – https://vk.com/gamio_ru

Discord – https://discord.gg/gvJFRz4f7b


Thanks for reading 🙂

Similar Posts

Leave a Reply