An alternative approach to combat malicious traffic
I'll tell you how to detect malicious traffic using a browser fingerprint.
I will also give a link where you can see a copy of your browser and any other bot program: those who wish can “play” at their leisure.
I, Grigory Melnikov, founder of the online service for detecting bots KillBot, continue the series of articles devoted to the problem of traffic quality.
Standard methods of traffic filtering
A black box with a set of rules and a probability number – filtering traffic on such paradigms is difficult. Transparency, flexibility, the ability to correct errors – this is what an antifraud that is easy to apply in practice should be based on.
Common methods for combating malicious traffic include the following:
Fingerprint Analysis visit to match real existing devices. This is an outdated method, since fingerprints of real devices are sold.
IP address analysis for spam. Outdated method, as bots use mobile proxies with IP rotation every 2-10 minutes.
Behavioural analysis. An excellent method if you use the fact of payment with real money as a behavior. Unfortunately, the targeting of the “solvent” audience in YAN ignores this obvious fact. Here is a case: how bots from YAN click on hidden links.
AI behavior analysis user on the site. Will not work for modern bots. AI solves captcha, which I'm stuck on: reproducing mouse movement over active elements of the site, data entry, etc. – a piece of cake.
Bots are like brothers – they are born from the same source
Therefore, the only real method of fighting bots is to construct (build) what unites all these brothers. If you find what unites them, then the entire network of bots will go under the filter.
Internet bots are united by the program that generates them. For example, users of the Opera browser can be considered “bots” – they all use the same browser, which unites them.
Read my other article: who generates bots, what software and why.
Every software on the Internet has its own copy – be it a browser, a bot, a checker or any other program for accessing a site.
To check the fingerprint of the site access program, there is a special checker – here it is: https://killbot.ru/snpsht.html
If you follow the link, you will see a number – this is a snapshot of your browser. At the same time, the snapshot of the bot program that automates the actions of the Chrome browser will be different, despite the fact that the bot can replace the fingerprint and introduce other de-identifying noises.
Watch a video demonstrating real browser and bot program snapshots:
Anti-fraud algorithm using browser snapshot approach
building a browser snapshot
let's see who the cast belongs to:
— if the snapshot matches the snapshot of the real browser, then the visit is processed as a visit from a real user;
— if the snapshot does not belong to the system browser, then we can classify it ourselves (for example, if among the snapshots there are facts of payment, then the snapshot should be attributed to real users);
— if the fingerprint is not present in known browsers and all visits to this fingerprint do not contain live conversions, then these are bots;
— if the imprint is in a pre-compiled list of bots, then it is a bot;
Thus, if we managed to build an entity – a “cast”, then this gives us unlimited control over traffic:
I can check the traffic behind the copy manually (live conversions + Yandex Webvisor)
Imprints for which there are no payments for a significant sample (possibly the same behavior) are bots.
visits with bot fingerprints can be processed differently: conversions can be ignored, blocked, redirected to another page, etc.
if suddenly I see both users and bots behind the snapshot, then this snapshot can also be processed differently: show a slider, captcha, 2FA verification, etc. depending on the goals.
It is clear that we will not find a fingerprint anywhere in the bot and browser code – this is mathematics. The browser parameters are folded by a hash function and a number is obtained. This hash function is resistant to parameter changes, fingerprint substitution, noise, etc. That is, we get the identifier of the program that generates bots. And this is a very good tool for filtering traffic within the site: I see fingerprints, I see visits, I see conversions – it is elementary to highlight bot visits by fingerprints with this approach.
In one of the next posts I will talk about constructing DEVICE_ID – a unique device identifier – it will be the same for the workstation, regardless of which browser was used to access the site: i.e. bots generated by one server will also have one DEVICE_ID identifier. The TOR browser may be an exception.
I will also give a case study on cleaning website traffic from bots (the largest virtual PBX provider) – this will be a very interesting case, so subscribe to my telegram channel so you don't miss it: https://t.me/KillBotRus
