Databases + Telegram Bot in C#. SKitLs Framework v.2

Good afternoon, dear readers!

Not so long ago I published an article about the express creation of a bot for Telegram on framework SKitLs.Bots.Telegram. Since then, the internal composition of the framework has changed significantly, however, preliminary versions have been released. *.BotProcesses And *.databases and second version *.Core.

In this article, I would like to give a more detailed look at the capabilities of the framework, as well as highlight the new features of the released extensions. In fact, the goal of this guideline will be to implement functionality similar to that written in the last article, but on the axes of two new extensions.

At the end of the article, after reading, I will express my opinion on the scope of new packages.

WeatherBot update

Everything new is well forgotten old.

Before updating to the second version, let’s recall one point from the last article. Localization of debug messages, which at that time we tried unsuccessfully to make the line:

BotBuilder.DebugSettings.DebugLanguage = LangKey.RU;

Since the update v2.0 project core, this functionality is available and allows you to localize debugging. In this example, I recreated the same error that occurred last time – I removed one of the event handlers. As a result, I received an error and a warning in Russian:

On new tracks.

Before moving on to the tasks at hand, let’s go over the list of changes that need to be made to the code written last time.

First, we’ll have to update some namespaces. It’s more of a purely mechanical thing.

Secondly, starting from version v1.3, the ArgedInteractions extension updated the name of the interface IArgsSerializeService (was: IArgsSerilalizerService).

And v1.1 AdvancedMessages allows you to quickly create an Inline menu with a serializer, automatically allowing this interface through the manager.

public async Task ActionFromUpdate(..., ISignedUpdate update)
{
  // ...
  //var res = new PairedInlineMenu()
  //{
  //    Serializer = update.Owner.ResolveService<IArgsSerilalizerService>(),
  //};
  var res = new PairedInlineMenu(update.Owner);
}

Thirdly, the interface has been updated IUsersManager. Appropriate Async tokens have been added to the methods. This does not entail changing internal processes, only the need to update the naming.

Well, on the little things: IAplicant now requires a method ApplyTo instead of ApplyFor; all shipping fixed from Delivery on Delivery.

Without leaving the checkout

Also, without changing our project with the possibilities of new extensions, we will make some changes, which, in my opinion, I did not pay enough attention to last time, and we will devote time to the visual component of the project.

First. Editing messages.

It annoys me a little that when I run the “Find” command, a new message is sent. First, it’s ugly: the message under the menu is not aesthetically pleasing, in my opinion. Secondly, it is unsafe: with more complex logic, it is better to prohibit the user from performing the same actions twice.

Let’s update the code as follows: use a special wrapper class EditWrapperto edit an existing message rather than posting a new one:

// async Task Do_SearchAsync()
// Удаляем: await update.Owner.DeliveryService.ReplyToSender(message, update);
await update.Owner.DeliveryService.ReplyToSender(new EditWrapper(message, update.TriggerMessageId), update);

I don’t attach a screenshot, because it looks unattractive, but the essence is completely conveyed by the code: the text of the message was rewritten, and the menu was deleted. The only thing left for the user is to enter the city or “Exit”.

Second. Access to ITelegramBotClient.

The next aspect. Visualization. The SKitLs framework, as already mentioned, is a kind of wrapper around the Telegram.Bot library. Derived classes are used to send messages. IDeliverySystem. But what if the default functionality of implementations does not allow you to send anything other than text messages? Yes, you could write your own class like WeatherDelivery : IDeliverySystem and fasten the necessary functionality, but there is an easier option.

For example, along with the weather in a city, I want to send the coordinates of the city as a geolocation. In this case, without a framework, one could turn to the entity ITelegramBotClient from the Telegram.Bot library and call the appropriate method SendLocationAsync(). The SKitLs framework also allows you to access this entity through the Bot property of the BotManager class (BotManager.Bot).

Having sent a message to the user, we will save the essence of the sent message and, having received the Id of the sent message, we will send the geolocation with a response to the sent message. It will all look like this:

// async Task Do_InputCityAsync()
// Удаляем: await update.Owner.DeliveryService.ReplyToSender(message, update);
var resp = await update.Owner.DeliveryService.ReplyToSender(message, update);
await update.Owner.Bot.SendLocationAsync(update.ChatId, latitude, longitude,
                                         replyToMessageId: resp.Message?.MessageId);

Third. “message box”.

The last point that I would like to demonstrate is the call functionality AnswerCallbackQuery classic Telegram client.

Just like in the previous example, this method is accessed via BotManager.Bot, however, calling it requires a callbackQueryId parameter. All existing update castes inherited from ICastedUpdateprovide access both to the original Telegram.Bot.Types.Update update that came from the server via CastedUpdate.OriginalSource, and to unpacked and typed data of a particular type. In this case, this property Telegram.Bot.Types.CallbackQuery Callback type updates SignedCallbackUpdatewhich gives access to the Id of this CallbackQuery.

// async Task Do_FollowGeocodeAsync(... SignedCallbackUpdate update)

// Меням этот код
// var message = new OutputMessageText(update.Message.Text +
//      $"\n\nГород сохранён в избранное!")
// {
//     Menu = null,
// };
// await update.Owner.DeliveryService.ReplyToSender(
//     new EditWrapper(message, update.TriggerMessageId), update);

// Удаляем меню сообщения, то есть кнопку "В избранное"
await update.Owner.Bot.EditMessageReplyMarkupAsync(update.ChatId,
    update.TriggerMessageId, null);
// Отвечаем на коллбэк в виде встроенного уведомления
await update.Owner.Bot.AnswerCallbackQueryAsync(update.Callback.Id,
    "Город сохранён в избранное!", showAlert: false);

More little things.

It is not related to the topic of conversation, but I will also note this point: we will add a couple of methods for converting coordinates from double to a beautiful form of the Axx°yy’zz” format. IN GeoCoderInfo add:

// GeoCoderInfo
// Здесь убираем "Город" перед {Name}
public string GetDisplay() => $"{Name} ({BeautyLatitude(Latitude)} {BeautyLongitude(Longitude)})";

public static string BeautyLatitude(double coordinate)
  => $"{(coordinate >= 0 ? "N" : "S")}{BeautyCoordinate(coordinate)}";
public static string BeautyLongitude(double coordinate)
  => $"{(coordinate >= 0 ? "E" : "W")}{BeautyCoordinate(coordinate)}";
public static string BeautyCoordinate(double coordinate)
{
    int degrees = (int)coordinate;
    double minutesAndSeconds = Math.Abs(coordinate - degrees) * 60;
    int minutes = (int)minutesAndSeconds;
    double seconds = (minutesAndSeconds - minutes) * 60;

    return $"{Math.Abs(degrees)}°{minutes:00}'{seconds:00.00}''";
}

And update Do_InputCityAsync

// async Task Do_InputCityAsync
var place = new GeoCoderInfo(cityName, longitude, latitude);
var resultMessage = $"Погода в запрошенном месте:\n{place.GetDisplay()}\n\n";
// ...
var resp = await update.Owner.Bot.SendLocationAsync(update.ChatId, latitude, longitude);
var message = new OutputMessageText(resultMessage)
{
    Menu = menu,
    // ReplyToMessageId = resp.MessageId
};
await update.Owner.DeliveryService.ReplyToSender(message, update);

Intermediate sump.

These actions do not really affect the abilities of the bot, however, they allow you to see the functionality of the framework more widely. All updated functionality is available in the same GitHub repositoriesin the branch [v1.0]-updated. Further in the article, we will deal directly with the study of new extensions and code completion.

Preparatory work

Before work, let’s open the NuGet manager, enable “Previews” and download the missing packages: SKitLs.Bots.Telegram.BotProcesses and SKitLs.Bots.Telegram.DataBases.

So, first of all, let’s turn to the possibilities of the project. *.DataBases and create an assembly method for our data manager (be careful before GetMenuManager())

// Program
private static IDataManager GetDataManager()
{
    var dm = new DefaultDataManager(databaseLabel: "Избранное [DM]");
  
    // Здесь будет заполнение
  
    return dm;
}

Creation of a dataset.

The data manager works with data of type IBotDataSet, which carry the functions of storing, changing and displaying arrays of displayed data. By default, this interface is implemented in two classes: BotDataSet<T> And UserContextDataSet<T>.

Both classes work with classes that support the interface IBotDisplayable (where T : class, IBotDisplayable).

And their differences lie in the implementation of the method GetContextSubset(ISignedUpdate): if the first class returns the entire amount of data, then the second class allows you to select from the general data set those data objects whose owner is the user who requested the opening of this dataset. Therefore, the second class introduces an additional restriction on the data with which it is able to work: interface support IOwnedData.

To summarize, then BotDataSet<T> will display generally all data of type available in the bot, while UserContextDataSet<T> will automatically fetch user related data before opening.

Since the data manager will store a list of all cities in general, entered by all users into the “Favorites” category, we will define this dataset as UserContext so as not to confuse users and display its list for each.

// GetDataManager()
var favorites = new UserContextDataset<GeoCoderInfo>("favorites", dsLabel: "Города");
dm.AddAsync(favorites);

We immediately get an error: for GeoCoderInfo missing two interfaces. Let’s go to the appropriate class and define them.

internal class GeoCoderInfo : IBotDisplayable, IOwnedData
{
    public long BotArgId { get; set; }
    public void UpdateId(long id) => BotArgId = id;
    // ...
    public bool IsOwnedBy(long userId) // => ...;

    public string ListDisplay() => Name;
    public string ListLabel() => Name;
    public string FullDisplay(params string[] args) => GetDisplay();
}

As for the method IsOwnedBywhich is used to define the UserContextDataSet, then it could be defined directly by introducing the long UserOwnerId property.

However, imagine 10,000 Muscovites, each of whom will keep Moscow in their favorites. That’s 10,000 entries containing identical information! Extremely inefficient.

We optimize as follows: we know for sure that the arrays of city geotags do not change – the city and its coordinates are static. Therefore, we will store a collection of owners of this label and return the sender’s belonging to it.

public List<long> Owners { get; } = new();
public bool IsOwnedBy(long userId) => Owners.Contains(userId);

Ideally, it would be necessary to divide all this into two classes like: GeoCodeArgument And GeoCodeData, each of which would be responsible for its own entity: one for storage in the database, the second for use as a callback argument. But, thank God, we do not have the task of making it perfect.

Dataset integration.

Now you need to update the functionality of adding to favorites. Since the user class no longer stores information about their favorites, and the geotag class stores information about owners, we will write the following:

// async Task Do_FollowGeocodeAsync()
//user.Favs.Add(args);
var geoCodes = update.Owner
  .ResolveService<IDataManager>()
  .GetSet<GeoCoderInfo>();
var code = geoCodes
  .Find(x => x.Longitude ==  args.Longitude && x.Latitude == args.Latitude);
if (code is null)
{
    code = args;
    await geoCodes.AddAsync(code, update);
}
code.Owners.Add(update.Sender.TelegramId);

In this code, we get the data manager service used in our bot, in which we search for a dataset with the data type GeoCoderInfo. Next, we search for a geotag by coordinates and add the sender to its owners. If such a label is not found, then create and add it.

Update the callback argument UnfollowGeocode With IntWrapper on GeoCoderInfo and similarly update the deletion logic.

// async Task Do_UnfollowGeocodeAsync(*GeoCoderInfo* args, ...)
//user.Favs.RemoveAt(args.Value);
var geoCodes = update.Owner
  .ResolveService<IDataManager>()
  .GetSet<GeoCoderInfo>();
var code = geoCodes
  .Find(x => x.Longitude == args.Longitude && x.Latitude == args.Latitude);
code?.Owners.Remove(update.Sender.TelegramId);

Update the logic for adding a callback UnfollowGeocode in the methods menu Do_OpenFollowAsync And Do_LoadWeatherAsync

// menu.Add(UnfollowGeocode, new IntWrapper(user.Favs.FindIndex(x => x.Name == args.Name)));
menu.Add(UnfollowGeocode, args);

For the method of obtaining user-saved geotags, you can update the class BotUserby adding a method to get the geotags the user owns.

// BotUser
public List<GeoCoderInfo> GetFavorites(ICastedUpdate update) => update.Owner
    .ResolveService<IDataManager>()
    .GetSet<GeoCoderInfo>()
    .GetUserSubset(TelegramId);

This method will give us access to the user’s favorites on any update. In particular, we update GetSavedList() And SavedFavoriteMenu.Build

// IOutputMessage GetSavedList()
var favs = user.GetFavorites(update);
if (favs.Count == 0) message += "Ничего нет";
foreach (var favorite in favs)
{
    message += $"- {favorite.Name}\n";
}

// SavedFavoriteMenu.Build()
foreach (var favorite in user.GetFavorites(update))
    // ...

Data manager connection.

Now that the functionality has been prepared, it is necessary to connect the data manager to the bot and organize access to the saved data.

// Program.Main()
var dataManager = GetDataManager();
var mm = GetMenuManager(dataManager);
// DataManager требует Stateful Callback
var statefulCallbacks = new DefaultStatefulManager<SignedCallbackUpdate>();
var privateCallbacks = new DefaultCallbackHandler()
{
    CallbackManager = statefulCallbacks,
};


var bot = BotBuilder.NewBuilder(BotApiKey)
    .EnablePrivates(privates)
    .AddService<IArgsSerializeService>(new DefaultArgsSerializeService())
    .AddService<IMenuManager>(mm)
    // Кроме того, понадобится менеджер процессов для DataManager
    .AddService<IProcessManager>(new DefaultProcessManager())
    .AddService<IDataManager>(dataManager)
    .CustomDelivery(new AdvancedDeliverySystem())
    .Build();

bot.Settings.BotLanguage = LangKey.RU;
dataManager.ApplyTo(statefulInputs);
dataManager.ApplyTo(statefulCallbacks);

await bot.Listen();

private static IMenuManager GetMenuManager(IDataManager dm)
{
    // ...
    mainMenu.PathTo(savedPage);
    mainMenu.PathTo(dm.GetRootPage());    // <- Добавить путь к базе данных
    mainMenu.AddAction(StartSearching);
    // ...
    dm.ApplyTo(mm);
    return mm;
}

So we’ve added a new navigation button to our menu that takes us to our database. I haven’t removed the old one yet, so that I can check the functionality.

Final touches

IBotDataSet setup.

In general, everything works. But we are not satisfied with the buttons “Add” (the functionality of adding occurs through the search process) and “Edit” (geotags are not editable). In addition, in one way or another, we need to add the “Find out the weather” callback.

To solve the first two nuances, we simply prohibit adding and changing through the properties of the dataset.

Add the action “Check the weather” (LoadWeather) is possible through the method AddAction() our dataset. The only difference that arises is that the method requires a callback with a DtoArg argument, while our method works with T itself. Let’s update the signature of the callback and method () and add it to the available dataset actions.

// IDataManager GetDataManager()
var favorites = new UserContextDataSet<GeoCoderInfo>("favorites", dsLabel: "Города");
favorites.Properties.AllowAdd = false;
favorites.Properties.AllowEdit = false;
favorites.AddAction(LoadWeather);

// В Do_OpenFollowAsync комментим //menu.Add(LoadWeather, args), исходя из того,
//     что в будущем этот функционал урезан в целом.

private static BotArgedCallback<DtoArg<GeoCoderInfo>> LoadWeather => ...
private static async Task Do_LoadWeatherAsync(DtoArg<GeoCoderInfo> args...)
{
    // вместо args получаем доступ к значению аргумента - args.GetValue()
}

In fact, we did the same thing that I wrote about earlier – demarcated the data object GroCodeInfo And GeoCodeArgument with the only difference being that a generic type was used for these purposes DtoArg<T>. In a good way, it would be nice to change all callbacks by introducing this argument everywhere in order to preserve the data exchange logic.

But we just enjoy the result of the work done.

Delete update.

It seems that everything is fine. However, you may notice that after loading the weather and the “Delete” button, the cross disappeared. This is due to the fact that we have two fundamentally different buttons in front of us: one is loaded from IDataManager, the second we registered ourselves. In principle, this can be detected by looking at the logs: the actions we are interested in are highlighted in white and mx actionId is different.

A natural question arises: if we abandon the previous actions (UnfollowGeocode) and move to the *.DataBases rails, then what will happen when the delete button is pressed?

And in this case, the worst suspicions come true: by default, the dataset completely erases the requested object. On the contrary, we just need to change the data of subscribers. Well, in order to fix this, you need to rewrite the “Remove” method of the dataset, which is used by default.

// private static BotArgedCallback<GeoCoderInfo> UnfollowGeocode
//   => new(new LabeledData("Удалить", "UnfollowGeocode"), Do_UnfollowGeocodeAsync);
private static TextInputsProcessBase<GeoCoderInfo> RemoveWithUnfollow
  => new TerminatorProcess<GeoCoderInfo>(IST.Dynamic(), Do_UnfollowGeocodeAsync);
private static async Task Do_UnfollowGeocodeAsync(TextInputsArguments<GeoCoderInfo> args, SignedCallbackUpdate update)
{
    // ...
}

TerminatorProcess is one of the processes provided by *.BotProcesses. It is a kind of link between input processes and action delegates. Its method of action is simple: immediately after launch, it causes its own destruction TerminateAsync(), without waiting for the next incoming update, and immediately calling the action that should happen at the time it completes. However, as with all processes TextInputsProcessBase<T>it needs arguments wrapped in TextInputsArguments<T>.

As for the interior Do_UnfollowGeocodeAsyncthen it looks like this.

// Также получаем датасет
var geoCodes = update.Owner.ResolveService<IDataManager>().GetSet<GeoCoderInfo>();

// Проверяем статус выполнения. Если удаление подтверждено, то обновляем ДС
if (args.CompleteStatus == ProcessCompleteStatus.Success)
{
    //user.Favs.RemoveAt(args.Value);
    var code = geoCodes.Find(x => x.Longitude == args.BuildingInstance.Longitude && x.Latitude == args.BuildingInstance.Latitude);
    if (code is not null)
    {
        code.Owners.Remove(update.Sender.TelegramId);
        await geoCodes.UpdateAsync(code, update);
    }
}

// Утилита, позволяющая получить статус из языкового пакета
var resultText = geoCodes.ResolveStatus(args.CompleteStatus, DbActionType.Remove);
var menu = new PairedInlineMenu();
menu.Add("Выйти", update.Owner.ResolveService<IMenuManager>().BackCallback);
var message = new OutputMessageText(update.Message.Text + $"\n\n{resultText}")
{
    Menu = menu,
};
await update.Owner.DeliveryService.ReplyToSender(new EditWrapper(message, update.TriggerMessageId), update);

It remains only to update the removal process in our DS. The final dataset setup looks like this.

var favorites = new UserContextDataSet<GeoCoderInfo>("favorites", dsLabel: "Города");
favorites.Properties.AllowAdd = false;
favorites.Properties.AllowEdit = false;
favorites.UpdateProcess(RemoveWithUnfollow, DbActionType.Remove);
favorites.AddAction(LoadWeather);
dm.AddAsync(favorites);

// В Do_OpenFollowAsync опять комментим убранный коллбэк
//menu.Add(UnfollowGeocode, args);

// В Do_LoadWeatherAsync придётся отказаться от сохранения информации Pagination
// либо же сменить аргумент на ObjInfoArg
var dm = update.Owner.ResolveService<IDataManager>();
var menu = new PairedInlineMenu(update.Owner);
menu.Add(dm.RemoveExistingCallback,
         new ObjInfoArg(dm.GetSet<GeoCoderInfo>(), args.DataId));
menu.Add("Назад", update.Owner.ResolveService<IMenuManager>().BackCallback);

Let’s check the work.

Let’s take a look at the debug data and make sure that the city “Moscow” has not disappeared from our saves, but we – as a subscriber – have disappeared from the list.

Saving data

Separately, it is worth noting that neither IDataManager nor its implementation of DefaultDataManager provide ways to save data, being only a virtual storage that provides an API for working with your database to the framework. Therefore, we also need to implement methods for loading/saving data.

The database can be, for example, SQL or any other system. We will write this functionality through the primitive Serizlize / Deserizlize methods of the Newtonsoft.Json library and store the data in a JSON file.

private static readonly object locker = new();
private static Task<List<T>?> LoadFromJson<T>(string dataName)
{
    var filePath = $"resources/database.{dataName}.json";
    if (!Directory.Exists(new FileInfo(filePath).DirectoryName))
        Directory.CreateDirectory(new FileInfo(filePath).DirectoryName!);

    lock (locker)
    {
        List<T>? res = null;
        if (File.Exists(filePath))
        {
            string json = File.ReadAllText(filePath);
            res = JsonConvert.DeserializeObject<List<T>>(json);
        }
        return Task.FromResult(res);
    }
}
private static Task SaveDataToJson<T>(List<T> data, string dataName)
{
    var filePath = $"resources/database.{dataName}.json";
    if (!Directory.Exists(new FileInfo(filePath).DirectoryName))
        Directory.CreateDirectory(new FileInfo(filePath).DirectoryName!);
    lock (locker)
    {
        string json = JsonConvert.SerializeObject(data, Formatting.Indented);
        File.WriteAllText(filePath, json);
    }
    return Task.CompletedTask;
}

We will collect a dataset based on the data of the document database and connect the database update to dataset change events.

// IDataManager GetDataManager()
var favsId = "favs";
var favorites = new UserContextDataSet<GeoCoderInfo>(favsId,
    data: LoadFromJson<GeoCoderInfo>(favsId).Result, // <- Считать данные
    dsLabel: "Города");
favorites.Properties.AllowAdd = false;
favorites.Properties.AllowEdit = false;
favorites.UpdateProcess(RemoveWithUnfollow, DbActionType.Remove);
favorites.AddAction(LoadWeather);
favorites.ObjectAdded += (i, u) => SaveDataToJson(favorites.GetAll(), favsId);
favorites.ObjectUpdated += (i, u) => SaveDataToJson(favorites.GetAll(), favsId);
favorites.ObjectRemoved += (i, u) => SaveDataToJson(favorites.GetAll(), favsId);
dm.AddAsync(favorites);

After working with the bot a little, I subscribed to a couple of cities and unsubscribed from the third one. The received data looks like this and loads fine after restarting the bot client.

Here it is worth adding a formal postscript that, if possible, we need to remove the load from the geocoder API and first try to load geotag data from our dataset. In addition, you can store the latest weather data and only access the weather forecast if the forecast is out of date.

Summing up

It may seem that for such a primitive logic used in this bot, the functionality of new extensions is redundant, and their use in general is irrational. I can’t refute this thesis in any way, however, in this example, the main functions of the extensions were analyzed. As for the scope…

As promised, I answer this question posed at the beginning of the article. In my opinion, using all the capabilities of the framework will allow creating AIS and CRM systems based on Telegram. I plan to describe what it is and how it is in the next article, reinforcing the arguments with examples of a specific case that is currently being worked on.

Like other sources, the results of the work are available in the GitHub repositoryin the “v2.0-Release” branch.