from idea to implementation of a cybersecurity system
Background: personal experience and first steps
I described my path in a previous article https://habr.com/ru/articles/813239/but in short, my career in the field of information security began, like many others, with work in the IT infrastructure. At first, my company focused on maintaining the stability of networks and systems for various companies, and regularly faced the typical problems associated with cyber attacks. Once, in a company where the information security staff was minimal, we implemented SIEM – a solution that was believed to radically improve security. However, this was an expensive and time-consuming process. SIEM not only did not live up to expectations, but also generated a bunch of incidents, most of which did not pose a real threat. Experts spent time analyzing many events that were, in fact, insignificant. With each new false positive, confidence in the system decreased. Eventually, people simply began to ignore the warnings, believing that the system was safe, although this was far from the case.
This is how I formulated the key SIEM problem:
In typical organizations, where the number of information security specialists is limited, the use of SIEM often does not lead to the expected results. And this is what opened my eyes to the need to find a new approach. I started thinking about how I could automate security processes without burdening the team with false alarms and complex settings.
Why SIEM-based automation doesn't work out of the box
In practice, it turned out that setting up a SIEM is another big headache. Each incident needs to be classified, to understand what is important and what is just noise. Building correlation rules for incidents is complex and time-consuming. When I started working with SOAR systems that were supposed to automate incident processing, it became clear that they only work in conjunction with SIEM, which means that before achieving SOAR technological maturity within the company, it is necessary to go through the difficult path of setting up and training SIEM with employees and infrastructure. At this stage, most organizations are faced with enormous difficulties, and the systems either are not fully configured or begin to overload the team with false incidents. When we faced the real challenges of implementing SIEM, it became clear: existing solutions are overloaded with incidents and require too many resources to configure.
At one of our work meetings, I shared these thoughts with my colleagues, and we began to think about how we could build a solution that would circumvent these problems. We remembered the old and well-known principle of fail2ban – if someone does suspicious actions, we block him; there is even a set of libraries with the same name. Of course, this decision seemed too rigid and one-sided to us, but this is how the idea of a new system was born, based on a similar approach, but in addition we needed: flexibility, learning ability, positive feedback.
Practice: developing a scoring system
Instead of blocking everyone at the first suspicion, we decided to use a real-time scoring system. We took as a basis the concept of two indices: a utility index and a danger index. We began to build a flexible system that could distinguish between real threats and random, benign actions.
Utility index: This parameter shows the extent to which the agent (user or device) performs useful actions on the network. Of course, setting it up is complex, since its usefulness depends on specific tasks and often the index parameters will have to be configured on the target system.
Hazard index: everything is much simpler here. Most systems automatically record suspicious actions by agents, such as attempts to log into non-existent pages, scanning network resources, etc. A danger index is accumulated for such actions.
Our key idea was that if the utility index is low and the danger index is growing, then we need to temporarily restrict access to such an agent. This would minimize the risk of intruders while simultaneously avoiding false positives.
It quickly became clear that we needed a “cooling down” mechanism, because we couldn’t ban everyone forever. We have implemented a TTL (Time To Live) system for both indexes. This means that danger and usefulness “cool down” over time – in particular, if an agent stops performing suspicious actions, its danger index decreases. For example, if a user accidentally logs in somewhere with an incorrect password, he will not be blocked immediately – the danger index will increase by only one for a few hours, but in the case of repeated and multiple actions it will continue to grow.
So, first theoretically and then practically, we developed a system with a detailed and dynamic assessment of the actions of agents. The introduction of two indices – usefulness and danger – allows flexible access control, minimizing false positives and maintaining high response accuracy. This solves the main problem of SIEM and SOAR – the excessive number of events and the complexity of setting up rules. Such a system, in fact, not only responds to incidents, but also learns from real activity and it turns out that the larger the protected perimeter, the more secure each element of the network is, because his defense is based on general statistics.
Case Study
One of the most striking cases of implementing our scoring system occurred in a project for a large e-commerce client. In the website security system, we began tracking the actions of users who tried to get into administrative areas through requests to pages such as /wp-admin, /phpMyAdmin, /bitrix and so on (here is an example regex: hpMyAdmin|wp-admin|admin| administrator|phpmyadmin|bitrix|typo3).
We set everything up this way: when an agent sends such requests and receives 404 from the server, we understand that it is looking for the admin panel, this is practically not a dangerous action in itself, and we will slightly increase the danger index for this agent. This allowed our system to flexibly manage access: if a user accidentally “stumbled” into the admin areas, then we did not block him, but if it was a targeted scan, then the danger index accumulated and access was limited.
And then everything went like clockwork: it turned out that all systems, in one way or another, report on how they work with the user, and even set criticality levels for their messages themselves, this allowed us to quickly accumulate a large base for signal sources. Systems, on the one hand, are smart, but on the other hand, they are completely deprived of the ability to exchange information with each other and they have no memory. We began to solve these two problems: now we accumulate scoring throughout the network, deep in time and differentiated by the degrees of threats.
We tested and implemented all these ideas in our own product, and tested it for a long time in our infrastructure and with partners. Less than a year ago we entered the market with ASoar — TDR (Threat Detection and Response) class system. The name Active SOAR underscores our philosophy: SOAR automates security orchestration, and the Active in the name indicates our commitment to proactively identifying and immediately responding to threats. Thus, we are at the intersection of TDR and SOAR systems, combining the best of them.
Another breakthrough – GHostHost
The HoneyPot concept has long been known in the cybersecurity world, but it is rarely used in practice, especially in corporate segments. Many people believe that setting traps for intruders is more of an academic interest than real protection. We decided to develop this idea and make it more practical.
Instead of having something at the end of the firewall rules that utilizes requests, like “deny all”we created a subsystem inside ASoar that we called GHostHost – a virtual machine with many HoneyPots. All requests that do not find their destination are redirected to this machine. As a result, we receive logs of all attack attempts and transfer them to ASoar for analysis. This approach not only helps us identify attackers at the reconnaissance stage, but also confuses them. Attackers see false targets, which makes their work less effective.
How does this work
Let's say an attacker is trying to scan our network by sending requests to various ports and IP addresses. Instead of these requests being blocked at the firewall level, they are redirected to GHostHost, a virtual server with a set of HoneyPots. This server keeps logs of all the attacker’s actions: what ports he scanned, what protocols he used, what login attempts he made, etc. These logs are then transferred to ASoar, where the system automatically evaluates the agent's actions and gives an assessment of their threat level.
Most importantly, thanks to this approach, an attacker cannot conduct reconnaissance of our real infrastructure, either inside the perimeter or outside. He hits false targets that seem real to him, and therefore wastes time, while our system manages to react and block him.
Case Study
At one of the sites, our specialists implemented GHostHost to protect the internal network. On the very first day of work, we recorded many scanning attempts from one of the machines where a careless administrator had installed a “free antivirus”. Due to the fact that all requests to non-existent addresses were redirected to GHostHost, the system managed to detect the source of the scan in time and isolate it.
Reporting
One of the key elements of a successful cybersecurity system is accessible and understandable reporting. I won’t even go into detail once again about the fact that in kibbez everyone really likes to show their importance with empty reports and make “paper security” instead of real one. ASoar provides capabilities not only for in-depth analysis of the trace of each agent and its actions, but also tools for data visualization that allow you to quickly assess the situation. For example, an administrator can easily see how the danger index increased or decreased for a particular agent, what events occurred, and how they affected the assessment of its activity.
ASoar reporting helps answer key questions:
How many agents were blocked in the last minute, hour or day?
How many logs are coming from each system?
Which correlation rules worked most often?
For ease of analysis, all data is presented on time diagrams, which simplifies infrastructure monitoring and timely identification of problems, such as loss of logs or failures in the operation of data sources. The system also allows you to automatically generate reports for regular presentation to managers or clients, which increases the transparency and controllability of security processes.
In general, there is still a lot of work there, so the ASoar reporting system is the most dynamically developing part and is constantly being improved, new ideas and features are constantly appearing there.
Market realities
In today's environment, companies are faced with a serious shortage of qualified cybersecurity specialists. Every year their deficit grows, and their services become more and more expensive. This is due to the fact that cyber threats are developing faster than companies can train new specialists. And as my friend said: “The period of calm ends and the World ceases to be a safer place.” As the complexity of attacks and networks grows, the workload on information security specialists increases, and the requirements for their qualifications become higher.
Such realities of the cybersecurity market create increased demand for solutions that can automate protection and minimize human participation in incident processing. In this regard, ASoar turns out to be the optimal choice for companies that do not want to maintain large information security teams. It replaces dozens of hours of manual work with automated data processing and real-time threat analysis, allowing you to significantly reduce security costs.
Consumer properties of ASoar
ASoar was created taking into account the real needs of companies seeking to optimize security costs and minimize the human factor. The system has the following consumer properties:
Automation: Most processes that require the participation of specialists in traditional SIEM systems are automated in ASoar. This can significantly reduce the workload on the security team and quickly respond to threats, while allowing administrators to sleep peacefully at night.
Flexibility and adaptability: ASoar easily integrates with existing security systems, collecting logs from various sources and providing universal solutions for companies at various stages of technological maturity.
Efficiency: Even companies with limited resources can receive a high level of protection without the need to maintain large teams of specialists. Thanks to the agent scoring model, the system allows you to accurately identify and block threats without overloading your staff with false positives.
Low level false positive: An important advantage of ASoar over its competitors is the low level of false positives. The system uses intelligent algorithms to analyze the actions of agents in real time, weighing the utility index and the danger index. This allows you to more accurately assess suspicious activity and minimize the number of false alarms. As a result, administrators do not waste time on unnecessary incident review and can focus on real threats. This is especially important for companies with limited information security resources.
Thus, ASoar allows you not only to reduce information security costs, but also to increase its level through process automation and the use of modern technologies, such as GHostHost and real-time agent scoring systems based on general network usage statistics.
ASoar now and plans for the future
ASoar currently provides powerful tools to automate and orchestrate cybersecurity processes. The GHostHost signaling subsystem allows you to further increase the sensitivity of ASoar, detect and stop intrusions at different levels. Our reporting system allows administrators to see how and why the agents’ danger index increased, what actions they performed, as well as monitor statistics on blocked agents and security rules. We provide timing diagrams for monitoring logs and their sources, which allows you to quickly identify problems in the infrastructure.
However, we understand that the development of the system should not stop there. We are already planning further development of interfaces and expanding the capabilities for integration with external security and management systems, planning the introduction of artificial intelligence to identify complex correlations and algorithms based on entropy calculations to identify anomalies (the implementation of anomaly detection index is planned).
Conclusion
The history of the creation of ASoar is the history of the search for new approaches in cybersecurity. We realized that traditional systems such as SIEM and SOAR cannot always provide sufficient protection. That's why we created a system that works in real time, focuses on automation and flexibility, allowing administrators to focus on real threats rather than false incidents.
Today, ASoar is not just a security system, it is the implementation of a comprehensive approach to security that helps companies improve their cybersecurity and reduce the cost of maintaining it. ASoar not only effectively filters agents based on their behavior, but also provides the necessary information to monitor and improve security processes. This makes the system attractive to both technical specialists and managers responsible for security in the organization.
And it is important in current realities to point out that ASoar was developed in Russia and is included in registry domestic software.
Acknowledgments
In this whole story, I represent not only myself, but also our entire team, and I want to express my deep gratitude to the people who made a huge contribution to the creation of ASoar. First of all, these are my colleagues, our full-time and part-time development team brought ideas to life, tested and brought the project to commercial launch. And I want to thank all my friends and colleagues from related companies, whom I invited to consultations and brainstorming sessions, who shared their experience and supported our product, helped to debug it, agreed to be the first users and gave constructive feedback!
