Telegram removed our ChatGPT bot for 27,000 users without explanation
A few weeks ago, a colleague and I made a ChatGPT-proxy bot in Telegram, which helped in our work chats and just entertained us. It’s written in TypeScript and the awesome Grammy framework, and uses the Open AI API, specifically the gpt3.5-turbo model used in the original ChatGPT.
A little about the bot itself
The bot itself is a simple bridge between Telegram and ChatGPT, while it remembers previous messages in the chat, using them as the context for the next question. In addition, the bot supports group chats. Moreover, we have Privacy Mode enabled in groups, so the bot sees only those messages that contain the /ask command.
We have added a lot of additional functionality, for example, you can send the bot a link to any website/article, or even a YouTube video and ask a question, which the bot will answer by adding subtitles to the context or the content of the site that the bot receives by rendering it in Puppeteer. In both cases, you have to put up with the context limit of 4096 tokens in GPT-3.5. I found an npm package as a solution ts-textrankwhich allowed us to summarize the contents of the text to the desired length, while losing as little information as possible.
We have also added the ability to recognize voice through the Google Cloud Speech API, but we plan to upgrade to a more powerful one. Whisper by OpenAI. By the way, this model can be raised even at home on the GPU, the project lies on GitHub. The largest version of this model, which supports all languages, takes up ~10 GB of video memory.
Where there is voice recognition, there is synthesis. In 2023, such things are done very simply, it is enough to install corresponding npm package and call 1 method. By the way, we plan to generate a Russian voice through Yandex Speech Kitwhich gives a much better result than the Google Cloud Text To Speech API.
As a result, our bot, in terms of knowledge of everything and everything, by a wide margin replaced Siri, Alice from Yandex and similar modern voice assistants, and all this in a couple of lines of code using 3 npm packages!
Finally, at some point, we taught the bot to generate pictures from a text description via DALL·E. The only problem is that this API only supports English. No problem! We always have a GPT that can accept instructions in Russian and instruct their fellow DALL·E in detail in English. In general, the possibilities of using GPT are endless, I’m afraid to imagine what will happen in the future.
A lot more has been added in the process. For example, setting the creativity of the bot’s answers, a set of ready-made instructions that set the role of the bot in the chat (Copilot, interviewer, business idea generator, gopnik and a bunch of others), you can even write such an instruction yourself using the /prompt command.
Deploy to production
At some point, we realized that our bot had scattered through the chats of our colleagues and friends, as well as their colleagues and friends. We decided to try to monetize this project, and began to prepare everything for launch:
Drawn and made beautiful landing page
Connected payment systems for recurrent payments
Deployed the entire infrastructure in Google Cloud using only Cloud Functons, Cloud Pub/Sub, Cloud SQL and Cloud Redis
Set up CI/CD via GitHub Actions
Prepared the bot for a heavy load by running Deployment Checklist in the Grammy documentation. It is very important to use webhooks instead of long-polling, set up a ratelimiter so that our bot is not spammed, and store sessions in Redis.
Set up Sentry to track runtime errors.
We threw a dashboard in Grafana to monitor the main metrics – the number of requests to the Open AI API, their status code, latency, and so on.
Added English
Implemented several paid plans with different restrictions and functionality
Achieved an increase in billing limits in Open AI to $3200 per month.
In general, the infrastructure turned out to be very fault-tolerant, minimalistic, predictable, scalable and relatively cheap (in fact, Google Cloud is free for us so far, since all accounts are given $300 upon registration there.)
Advertising
In the first days, we got about 5,000 users by buying a couple of posts in Telegram publics, as well as setting up a target in Yandex. The demand for this bot is huge. Those who bought a subscription used it literally every day.
A week later, we bought advertising in the Telegram public for 3.7 million subscribers and ~ 800 thousand post views. This brought us another 22,000 users.
Fiasco
A little more than a day later, the bot simply disappeared, as if it had never existed. In BotFather, he also disappeared. You can not count on Telegram support, since, as you know, the messenger simply does not exist. Now we are in a complete misunderstanding of what to do next, how to develop any projects in Telegram at all, if at some point your business may simply disappear without any warnings or explanations. At the moment, we have launched a bot at a new address gpt_askbotwe hope that we will return access to the original GPTaskBot.
There are suspicions that someone convinced our bot to send such a response, which can be easily complained about. The problem is that we can’t put any restrictions on the bot’s responses, since it’s a black box and there are a huge number of jailbreaks currently available to bypass ChatGPT’s built-in censorship. Yes, there is a Moderation API, but unfortunately it only supports English.
If someone has ideas what can be done in this situation, write in the comments, we will be very grateful to you.
