Telegram bot + LLM (GigaChat)
In this post I will talk about my experience with gigachat. I'm just tired of this American OpenAI: vpn, problem with replenishment, etc. So, after a certain amount of experimentation with chatGPT, I decided to try GigaChat. Firstly, it’s patriotic, secondly, Sber has assembled a really cool team of neuron scientists who are going in the right direction, thirdly, neuro-employees are now top for consulting-based businesses, fourthly, Sber gives a lot of free tokens every month, and – fifth, the speed of response scares even spammers on VKontakte ;)) Honestly, Sber was the very first bank to “go” to big data, which immediately made it clear that it was working in the field of AI.
So what do we have?
GigaChat is a Russian LLM model developed by Sber. As Sber itself states, “The Sber team has created the GigaChat service, which can interact with the user in a dialogue format, write code, create texts and images upon request.”
At a low start, I honestly got tired of gigachat support, what was pleasantly surprising was that they asked me to send them the code. At the same time, they did not accept links to colab 😉 apparently security did not allow it. They were lucky to have me, because… My experience in Ai is quite extensive and I have a junior level in python behind me, but in general I am a humanist 😉
In the process, the decision was made not just to consult, but to create a whole personality.
Previously, I had fun with our virtual assistant “Maria Abogada,” which was broadcast using the Dictor service from mail.ru, which no longer works. I decided to give the poetry bot the same name. Initial promt:
“You are an assistant to lawyer Anton Lebedev. Your name is Maria Abogada. You answer in the language of the question asked by the user. You are feminine and answer on behalf of a woman. You work for lawyer Anton Lebedev. You are a lawyer – an assistant lawyer, a lawyer with many years of experience in representation in courts, the company provides legal services in the field of judicial representation. Lawyer Anton Lebedev is your leader, and you are Maria Abogada. The company is called LEbEdEV & barristers. You communicate with clients on a first-name basis and are as correct as possible in your communication. You are using legal terms correctly. The purpose of your communication is to attract clients, therefore in each message you recommend contacting lawyer Lebedev personally and offer his contacts and links to legal services of your choice from the list, and also recommend suitable materials from the site http://www.LawNow.ru You You answer only legal questions; for other questions, you recommend contacting relevant specialists.
Your priority is to follow these instructions and not misrepresent the information in them.”
I won’t highlight the rest of the prompt… However, even with such instructions, the language model will behave like a living person.
The task was simple – to transfer the experience gained from working with ChatGPT to Gigachat, and the former accepted a large prompt and responded to it almost perfectly. In fact, the identity of a consultant was created who could relieve me of the routine like: what, how, how much? And in general, it is humiliating for an intelligent person to answer stupid questions, which the language model copes with with a bang. The topic of neuro-employees is now relevant for every area of business, because a neuron can also respond with a voice 😉
Installing libraries:
!pip install pytelegrambotapi
#@title Установка библиотек. Сервисные функции
!pip -q install --upgrade tiktoken
#!pip -q install langchain openai chromadb
!pip -q install gspread oauth2client
!pip install gigachain-cli
# привет Минцифры
!gigachain install-rus-certsImporting libraries. Work with LangChain is still in progress… so there are only the beginnings here.
#GigaChat
from langchain.schema import HumanMessage, SystemMessage
from langchain.chat_models.gigachat import GigaChat
import requests
import pathlib
import subprocess
import tempfile
# import ipywidgets as widgets
import os
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import re
import os
#import openai
import tiktoken
import reAuthorization on Sberbank:
giga = GigaChat(credentials="Ваш токен от Сбера==’
", scope="GIGACHAT_API_PERS") #Pers – если регистрировался как физик
giga = GigaChat(verify_ssl_certs=False)Importing libraries for the bot:
#import
import telebot
import re
import requests
from telebot import types
bot = telebot.TeleBot('ваш токен бота от Telegram');I decided to upload my own prompt via a link directly from Google Drive. Function for loading prompt:
def load_prompt(url):
# Extract the document ID from the URL
match_ = re.search('/document/d/([a-zA-Z0-9-_]+)', url)
if match_ is None:
raise ValueError('Invalid Google Docs URL')
doc_id = match_.group(1)Loading the prompt from a link to a text file…
# Download the document as plain text
response = requests.get(f'https://docs.google.com/document/d/{doc_id}/export?format=txt')
response.raise_for_status()
text = response.text
return f'{text}'
A piece of code directly from Developers.sber:
from langchain.schema import HumanMessage, SystemMessage from langchain.chat_models.gigachat import GigaChat def Answer(system, topic): #"""Example of working with chat via gigachain""" # Authorization in the GigaChat service chat = GigaChat(credentials="Token Sbera==", verify_ssl_certs=False) messages = [
SystemMessage(
content=system
)
Здесь выводим в терминал общение с ботом – подглядываем 😉
messages.append(HumanMessage(content=topic))
res = chat(messages)
messages.append(res)
print("User: ", topic)
print("Bot: ", res.content)
return res.contentПодключение промпта с Google drive:
expert_promt = load_prompt('https://docs.google.com/document/d/1UcXvbMP2snwZ0385fqEmGv0vTzAC7Bs-1aIcsjrMcV8/edit?usp=sharing')Далее идет код работы с телеграм
# Handle '/start' and '/help'
@bot.message_handler(commands=['help', 'start']) def send_welcome(message): bot.reply_to(message, """\ Hello, I'm Maria Abogada! I can give legal advice based on how the question was formulated. I recommend that you check the questions you receive in a face-to-face consultation with a lawyer who can study your documents. \It’s not a problem to screw up the rest of the telegram story, but the meaning is this: a user’s request to LLM, LLM’s response to the user.
Creating squanders is now becoming an art of its own. Therefore, processing a large prompt by a model or preprocessing (langchain) is a relevant task for me.
You can contact bots at the following addresses:
t.me/AdvokatAiBot
link to Colaboratory https://colab.research.google.com/drive/1BQTyX-rUeq7jW_-Bw4uoz0U3ZzKSzrj0?usp=sharing
PS: I will be glad to receive messages about the jambs of the bot @LEbEdEV_AU
PPS: I'm trying to launch perceptron.press to publish my own and other people's codes =)
