Setting up monitoring and tracking changes on sites

It so happened that within the framework of work and everyday tasks, I had a treasured list of resources that I have to regularly monitor “by hand”: I went to a site, looked to see if anything had changed, and closed it.

I think many have encountered such a routine and thought about automating actions in the browser. So I decided to find a convenient, and preferably free, tool that can perform my tasks:

news monitoring by keywords
checking organizations in open registries
monitoring prices and discounts
flight ticket tracking
monitoring the appearance of new announcements and vacancies, etc.

After a thorough search, I managed to find two free options: browser extension open source and cloud servicewhere they promise a free plan forever for a small number of checks. Both tools have proven to be quite popular, intuitive, and convenient. Let's see them in action.

Browser extension

In order for the monitoring to work around the clock without interfering with my use of the main PC, it was decided to install the browser extension on a separate virtual server (VPS).

For comfortable work, a VPS with 2 cores, 4GB RAM and the Ubuntu 24.04 operating system was rented. By default, the server is issued with root access without a graphical interface, so first we will create a new user (for example myrobot) with a strong password and install the desktop:

root@vps:~# adduser myrobot
root@vps:~# apt update && apt install ubuntu-desktop-minimal -y

To be able to connect to the virtual server desktop remotely, install the XRDP package:

root@vps:~# apt install xrdp -y

Now you can access our server using the standard “Remote Desktop Connection” in Windows, or using any XRDP client for Linux, for example Remmina (installed by default in desktop Ubuntu).

We connect, enter the username and password:

Launch the FireFox browser and install the Automa extension directly from addon catalogor by downloading it from GitHub. The same can be done for Google Chrome browserif you prefer it.

Working with Automa is extremely simple – all actions are modeled using action blocks connected by arrows. As an example, let's try to create a simple robot that will monitor my karma on Habr every day and send a message to Telegram when it grows. Let's see:

The whole process took a few minutes, below is a detailed description of the steps:

Setting up karma monitoring in Automa

Step 1: Set up a launch schedule
We launch the Automa plugin in the browser, create a new task, and in the “Trigger” block we set up its daily start.

Step 2. Create a new tab and open the user page
To open a page in a new tab, find the “New Tab” block on the left and drag it to the project we created. In the block settings, specify the URL of the user's page where we will view karma.

Step 3. Add a condition to check the karma level
Using the “Conditions” block and the built-in element selector, we get the karma value from the user's page, and also set the condition for further sending of the notification (if karma >30).

Step 4. Setting up sending messages to Telegram
This step is the most labor-intensive, as it involves creating your own telegram bot to send messages via the API using the “HTTP request” block.

If you have done this before, then from the list of actions below it will be enough to perform only the last point. In general, you need to do the following:

Launch BotFather in your Telegram and to create a bot, enter the command /newbot
Give your bot a name and choose a username for it. The name must end in “bot” and will be used in Telegram as @NAME bot, as well as a link for quick access to it: https://t.me/NAMEbot
If you get an error message, please choose another name as most names are already taken.
BotFather will send a message about the bot creation with a link to it and a secret token.
Open the link in your browser https://api.telegram.org/botXXX:YYYY/getUpdateswhere instead of XXX:YYYY specify the secret token of the created bot. In response, you will see something like: {"ok":true, "result":[]}
Add the created bot to your Telegram by following the link https://t.me/NAMEbotand write him any message, for example “Hello!”
Open (or refresh) the link from step 4 again, and in the new response you receive, find the chat ID. It should be here: "chat":{"id":xxxxxxxxxx,
Now, to send a message to Telegram, we just need to insert our secret token and chat ID into the link: https://api.telegram.org/botXXX:YYYY/sendMessage?chat_id=xxxxxxxxxx&text=Karma is more than 30. Congratulations!
In our project, this link should be inserted into the “HTTP request” block, specifying the request method as “GET”.

Step 5. After each launch, we will close the browser tab
Add the “Close tab” block to the project and link it with two arrows to the “Conditions” block, which checks the karma level. Now after each check, regardless of the karma value, the browser tab will close.

Step 6: Add a variable to stop checking after the notification is sent
Once the karma reaches 30, we will receive a message about it. But if we do not stop the task, the messages will continue to be sent daily.
To avoid this, we introduce an additional variable $$karma_checker_stoppedand we will check it every time the task is launched.
By default, the value of the variable will be set as 0and when the message is sent, it will change to 1.

Step 7. Testing: for our user, and another one with karma >30
First, let's manually run the task for our user (kaspra), but due to the low level of karma, the notification is not sent. For testing purposes, we will replace the user with a random one, with a karma higher than 30 (writer), and run the task again. As you can see, the message has been successfully delivered.

Advantages of Automa:

free and easy to use open source software
your own clean IP is used, which will avoid the appearance of captchas and the triggering of anti-bots
can be used not only for monitoring, but also for emulating frequently repeated actions in the browser
built-in task scheduler, integration with Google services, Javascript block for inserting arbitrary code, and many other features

Flaws:

For continuous monitoring, you need to keep your computer on. This can be solved by ordering an inexpensive virtual server
it takes a little time to see how the work with data storage is organized (variables and tables) and to “get your hand in” with the syntax of their writing. There is a large knowledge base
At the moment, unstable operation of the action recording mode (the ⦿ icon in the extension menu) has been noticed. The bug has already been reported reported for developers

Cloud service

The service we are going to talk about is called Browse AI. As part of the free tariff, we will try to set up karma monitoring with its help:

As a result, whenever there is any change in the karma value, we will receive a notification by email.

Advantages:

simple and clear
free for a small number of checks
integrations with Google Sheets, Zapier, Webhooks, etc.
API access

Flaws:

limited functionality, mainly intended for change tracking and data extraction
The created tasks are launched on Browse AI servers, the network addresses of which may be well known to anti-robots. For example, I had difficulty loading the Skyscanner website

Conclusion

Although both solutions considered are good, I prefer the browser extension, which successfully solves all the tasks. As for Browse AI, even in the considered example of karma monitoring, I was unable to implement the same logic of checking it and sending notifications to Telegram as in Automa.