Ask a bot to write a bot

In this article I want to share an interesting experience with ChatGPT, in which
a telegram bot was written in Python using the OpenAI API for
receiving responses to inquiries.

It all started with the fact that I wanted to register an account on OpenAI and understand who this ChatGPT of yours is. A little dancing with a tambourine, and the desired goal was achieved. I liked the bot, but there were some inconveniences due to VPN access. Then I got the idea to create a telegram bot that will send requests to the OpenAI API and return responses.

I spent the next hour reading documentation on the OpenAI API, but my laziness turned out to be more resourceful, and I decided to give ChatGPT the task of writing a bot myself.

What came out of it:

Beginnings of cyberpunk

TK text for ChatGPT

Greetings my dear! I have a task for you: write
telegram bot in Python that will send requests to the OpenAI API and
return answers. To work with the Telegram API, let’s use
telebot library

  1. Telegram bot must have two command handlers – /start and /help. The text of the response to commands can be anything.

  2. The bot should immediately send a response that the request has been accepted for work when it receives a request from the client.

  3. When the bot receives a request from the user, you need to call the create method on the Completion class using the OpenAI library. We set the text-davinci-003 model and max_tokens = 3500 as parameters.

  4. We then receive a response from the OpenAI API and return it to the user.

ChatGPT received a task from us and set to work. It is worth noting that the provided code will be used without changes and refactoring.

Here is the code it generated:

import telebot
import openai
import os

# устанавливаем ключ API OpenAI из переменной окружения
openai.api_key = os.environ.get("OPENAI_API_KEY")

# создаем экземпляр телеграм бота
bot = telebot.TeleBot('YOUR_TOKEN')


# создаем обработчик команд
@bot.message_handler(commands=['start'])
def start(message):
    bot.reply_to(message, "Привет! Я бот, который может помочь вам с OpenAI API.")


@bot.message_handler(commands=['help'])
def help(message):
    bot.reply_to(message,
                 "Вы можете отправлять запросы в OpenAI API через меня. Просто напишите мне свой запрос и я отправлю его на обработку.")


# создаем обработчик сообщений
@bot.message_handler(func=lambda message: True)
def echo(message):
    bot.reply_to(message, "Запрос принят в работу.")
    try:
        response = openai.Completion.create(
            engine="text-davinci-003",
            prompt=message.text,
            max_tokens=3500
        )
        bot.reply_to(message, response.choices[0].text)
    except:
        bot.reply_to(message, "Произошла ошибка при обработке вашего запроса.")


# запускаем телеграм бота
bot.polling()

We slightly correct this section of code bot = telebot.TeleBot('YOUR_TOKEN')so that the value is pulled from the environment variables.

First message to the bot

Voila! Literally one edit and it works. But we want to continue the conversation about the rain and clarify some more points, and we get something strange.

We continue the conversation about the rain, but do not mention it explicitly

The problem here is that the bot is not able to take into account the context of the conversation, since each new message is sent to the API separately. After reviewing the documentation, I realized that there is no explicit solution to this problem. (If I’m wrong, please correct me).

So I decided to make a crutch. We need to create some kind of cache and store information in it. We can conditionally split the cache into two levels: one will be stored in the database, and the other in runtime.

Based on this information, we are ready to make some changes to the original TOR.

Text of TK No. 2 for the bot

The task requires making changes to the bot code so that it can save
context of users’ previous messages and use it when
sending requests to the OpenAI API.

  1. You must use both an array and a SQLite database to store the context. The bot should first try to get the context information from the array. In this case, the array should be cleared after 10 minutes. If the context is not found in the array or its storage time has expired, the bot must access the database.

  2. It is important to provide for the possibility of notifying the user in case of errors.

    Thus, appropriate changes must be made to the bot code so that it can work with the context of users’ messages and provide secure data storage.

In the same text, we send our existing code to the bot so that it rewrites it.

We get the following result from the bot:

import telebot
import openai
import os
import sqlite3
from datetime import datetime, timedelta

# устанавливаем ключ API OpenAI из переменной окружения
openai.api_key = os.environ.get("OPENAI_API_KEY")

# создаем экземпляр телеграм бота
bot = telebot.TeleBot(os.environ.get("TG_API_KEY"))

# создаем подключение к базе данных
conn = sqlite3.connect("example.db", check_same_thread=False)

# создаем таблицу в базе данных для хранения контекста
with conn:
    cur = conn.cursor()
    cur.execute("CREATE TABLE IF NOT EXISTS context (user_id TEXT, message TEXT, timestamp TEXT)")

# задаем интервал, через который массив с контекстом будет очищаться
CONTEXT_CACHE_INTERVAL = timedelta(minutes=10)

# словарь, в котором будут храниться последние запросы пользователя
context_cache = {}


# создаем обработчик команд
@bot.message_handler(commands=['start'])
def start(message):
    bot.reply_to(message, "Привет! Я бот, который может помочь вам с OpenAI API.")


@bot.message_handler(commands=['help'])
def help(message):
    bot.reply_to(message,
                 "Вы можете отправлять запросы в OpenAI API через меня. Просто напишите мне свой запрос и я отправлю его на обработку.")


# создаем обработчик сообщений
@bot.message_handler(func=lambda message: True)
def echo(message):
    # смотрим, есть ли контекст в кэше
    if message.chat.id in context_cache and datetime.now() - context_cache[message.chat.id]['timestamp'] <= CONTEXT_CACHE_INTERVAL:
        context = context_cache[message.chat.id]['message']
    else:
        # если контекста в кэше нет, ищем его в базе данных
        with conn:
            cur = conn.cursor()
            cur.execute("SELECT message FROM context WHERE user_id = ? ORDER BY timestamp DESC LIMIT 1", (str(message.chat.id),))
            row = cur.fetchone()
            context = row[0] if row else ""

    bot.reply_to(message, "Запрос принят в работу.")
    try:
        response = openai.Completion.create(
            engine="text-davinci-003",
            prompt=context + message.text,
            max_tokens=3500
        )
        bot.reply_to(message, response.choices[0].text)

        # сохраняем контекст в кэше и базе данных
        with conn:
            cur = conn.cursor()
            cur.execute("INSERT INTO context (user_id, message, timestamp) VALUES (?, ?, ?)", (str(message.chat.id), context + message.text, datetime.now().strftime("%Y-%m-%d %H:%M:%S")))
            conn.commit()
        context_cache[message.chat.id] = {'message': context + message.text, 'timestamp': datetime.now()}

    except Exception as e:
        bot.reply_to(message, f"Произошла ошибка при обработке вашего запроса: {str(e)}")


# запускаем телеграм бота
bot.polling()

The program has successfully run and we are checking for a database with three columns. We continue to chat with the rain-themed bot and see that it has begun to look like a conversation with artificial intelligence, and not just like a random text generator.

Rain question with context

Unfortunately, this solution can have a side effect. If you have a long conversation on different topics, then there is a risk that the bot can answer all your previous questions, even if the last message had nothing to do with them. Although this effect is not always observed.

Side effect of crutches

This is because we are passing the current request along with the context in the request. And the context can contain a lot of information that is not relevant to the current request. We can clear the context each time, but this approach can be inefficient. It might be worth adding a separate command on the client side that will clear the context.

response = openai.Completion.create(
            engine="text-davinci-003",
            prompt=context + message.text,
            max_tokens=3500
        )

We continue to have a casual conversation with the bot, when suddenly we get an error: Произошла ошибка при обработке вашего запроса: This model's maximum context length is 4097 tokens, however you requested 4654 tokens (1154 in your prompt; 3500 for the completion). Please reduce your prompt; or completion length.

This is due to the limit on the number of characters in the request, which is set in the OpenAI API. And we still save the context in the cache. Therefore, you can still ask the bot to fix the error, and it gives us this code:

@bot.message_handler(commands=['drop_cache'])
@restricted_access
def drop_cache(message):
    user_id = message.from_user.id

    conn = get_conn()
    cursor = conn.cursor()

    cursor.execute('DELETE FROM context WHERE user_id=?', (user_id,))

    hot_cache.clear()

    conn.commit()
    bot.send_message(user_id, "Cache dropped.")

Here we are clearing the cache for a specific user_id to meet the api limits.

As a result, we get a working solution. It can be further improved and customized. Bot source code is available here.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *