creating a bot for monitoring logs
Recently, at work, I was asked to come up with a work-related task for students. Since I work on an infrastructure team, my daily tasks are unlikely to fit into their homework or term papers. To find a suitable idea, I started to look through the tools that my team and I frequently use. Most of them are integrated with chats and bots, and one of the key tools is Alert Bot. It monitors logs and sends notifications if something unusual happens. This allows us to detect and resolve incidents faster.
When I came up with this task for students, it occurred to me that such functionality could be useful to many. Perhaps someone could adapt this solution to their needs and make their life easier by automating the process of monitoring and responding to incidents. Actually, this is what I would like to share in my article.
Description of the task
So, students are asked to develop a bot that will analyze server logs and send notifications to Telegram if it detects errors or other anomalies. The main requirements for the project:
Log monitoring: The bot should periodically check the specified log file and look for errors in it.
Filtering data: It is necessary to filter out important messages (for example, with the ERROR or CRITICAL level) and ignore less significant ones (DEBUG, INFO).
Sending notifications: When errors are detected, the bot should send a message to the Telegram chat.
Configurability: Settings such as log path, check frequency and chat ID should be easy to change.
Solution architecture
Our bot will consist of the following main components:
Log file: A file containing server event logs that the bot will analyze.
Log analyzer: A module that reads a log file and extracts important messages from it.
Telegram bot: A module that sends found errors to the specified chat.
These components must interact with each other to ensure continuous monitoring of logs and prompt notification of problems.
Generating logs
Let's start by simulating server activity so that we have logs to analyze. We'll create a Go script that will generate logs with different message levels (e.g. INFO, WARNING, ERROR) in random order.
Let's assume that our “server” will periodically generate events with different levels of importance. To do this, we will write a script that will write logs to a file.
1. Create a function to read and write lines to a file
First, let's create functions that will read logs and write them:
func readLines(fileName string) ([]string, error) {
file, err := os.Open(fileName)
if os.IsNotExist(err) {
return []string{}, nil
} else if err != nil {
return nil, err
}
defer file.Close()
var lines []string
scanner := bufio.NewScanner(file)
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
return lines, scanner.Err()
}
func writeLines(lines []string, fileName string) error {
file, err := os.Create(fileName)
if err != nil {
return err
}
defer file.Close()
writer := bufio.NewWriter(file)
for _, line := range lines {
fmt.Fprintln(writer, line)
}
return writer.Flush()
}2. Functions for generating logs
Now let's create functions that will directly generate the logs for us.
var (
logLevels = []string{"DEBUG", "INFO", "ERROR"}
mu sync.Mutex
)
func generateLog() {
mu.Lock()
defer mu.Unlock()
lines, err := readLines(logFileName)
if err != nil {
fmt.Println("Error reading log file:", err)
return
}
if len(lines) >= maxLines {
lines = lines[len(lines)-maxLines+1:]
}
logLine := fmt.Sprintf("%s [%s] %s\n", time.Now().Format(time.RFC3339), logLevels[rand.Intn(len(logLevels))], generateRandomMessage())
lines = append(lines, logLine)
err = writeLines(lines, logFileName)
if err != nil {
fmt.Println("Error writing to log file:", err)
}
}
func generateRandomMessage() string {
messages := []string{
"User logged in",
"File uploaded",
"Error processing request",
"User logged out",
"Database connection established",
"Invalid input received",
}
return messages[rand.Intn(len(messages))]
}3. The main function for generating logs
Now let's put everything together in a main function that will periodically generate and write log messages:
const (
logFileName = "logs.log"
maxLines = 200
)
func main() {
rand.Seed(time.Now().UnixNano())
go func() {
for {
generateLog()
time.Sleep(time.Millisecond * 100)
}
}()
select {}
}It is important to remember that logs should not be stored indefinitely. Over time, their number can increase significantly, which will lead to an increase in the file size and possible performance issues. To avoid this, you should set reasonable limits on the number of records stored. In this example, we limit ourselves to storing the last 200 logs.
Continuing development of the bot in Go
Now that we have the generated logs, let's start developing a bot that will analyze them and send notifications to Telegram. Below are the basic steps for creating such a bot.
1. Installing the required libraries
To work with Telegram and analyze logs, we will need several external libraries. In Go, this can be done using go get:
go get -u github.com/go-telegram-bot-api/telegram-bot-api/v5
This library will provide us with an interface to interact with Telegram.
2. Reading and analyzing logs
The first step in our bot is to read the logs and find errors in them. We will use a function to read the file and filter out important messages.
package main
import (
"bufio"
"fmt"
"os"
"strings"
"time"
)
func readLogs(logFilePath string) ([]string, error) {
file, err := os.Open(logFilePath)
if err != nil {
return nil, err
}
defer file.Close()
var importantLogs []string
scanner := bufio.NewScanner(file)
for scanner.Scan() {
logLine := scanner.Text()
if strings.Contains(logLine, "ERROR") || strings.Contains(logLine, "CRITICAL") {
importantLogs = append(importantLogs, logLine)
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
return importantLogs, nil
}
3. Sending notifications to Telegram
The next step is to send notifications to Telegram. For this, we will use the library telegram-bot-api.
package main
import (
"log"
"github.com/go-telegram-bot-api/telegram-bot-api/v5"
)
func sendTelegramNotification(bot *tgbotapi.BotAPI, chatID int64, message string) {
msg := tgbotapi.NewMessage(chatID, message)
_, err := bot.Send(msg)
if err != nil {
log.Printf("Error sending message: %v", err)
}
}
4. Main function for monitoring logs and sending notifications
Now let's put it all together in the main function. It will periodically check the logs and send notifications if errors are found.
package main
import (
"fmt"
"log"
"time"
"os"
"github.com/go-telegram-bot-api/telegram-bot-api/v5"
)
func main() {
logFilePath := "server.log"
botToken := "YOUR_TELEGRAM_BOT_TOKEN"
chatID := int64(YOUR_CHAT_ID)
bot, err := tgbotapi.NewBotAPI(botToken)
if err != nil {
log.Panic(err)
}
log.Printf("Authorized on account %s", bot.Self.UserName)
for {
importantLogs, err := readLogs(logFilePath)
if err != nil {
log.Printf("Error reading log file: %v", err)
continue
}
for _, logMessage := range importantLogs {
sendTelegramNotification(bot, chatID, logMessage)
}
time.Sleep(10 * time.Second) // Проверяем логи каждые 10 секунд
}
}
5. Configurability
For convenience, you can put parameters such as the path to logs, bot token, and chat ID in a configuration file or use environment variables. This will simplify the adaptation of the bot to different operating conditions.
Example of using environment variables:
package main
import (
"os"
"log"
)
func main() {
logFilePath := os.Getenv("LOG_FILE_PATH")
botToken := os.Getenv("TELEGRAM_BOT_TOKEN")
chatID := os.Getenv("TELEGRAM_CHAT_ID")
if logFilePath == "" || botToken == "" || chatID == "" {
log.Fatal("Missing necessary environment variables")
}
// Остальной код...
}
Now that the bot is ready, it can be deployed to a server or cloud and used to monitor logs in the wild. Experienced developers are probably already familiar with such tasks, but I hope this article was useful for those who are just starting out in programming, working on their pet projects, or taking their first steps in the industry. Building such tools is a great way to gain skills that will come in handy in the real world.
