And the whole WAF is not enough: how we upgraded the website protection service

Hello! My name is Kirill, and at the DataLine cyber security center I am developing web application protection service (WAF): I communicate with information security and IT specialists from the client, find out their tasks, and am responsible for the correct operation of the service. In less than a year of setting up WAF, I was convinced that if you have a site, it will be attacked. And not always what you think.

We refined the solution several times: we studied attack vectors and attacker behavior better, added and configured new defenses, improved client interaction rules. I’ll tell you how our service developed on the basis of FortiWeb and what you need to take care of to protect your web application.


Service background

For our own sites, we once chose FortiWeb from Fortinet, we already knew the intricacies of working with it. Before my eyes was the WAF-as-a-service experience in the cloud at Fortinet.

In the West, the self-service model is widespread for WAF. The client is given a dedicated instance for self-configuration in the desired cloud (AWS, Azure and GCP), of course, not in the Russian Federation. This model has its own difficulties:

  • A client-side information security specialist must deeply understand the security features of web applications.
  • The client’s specialist needs to know the capabilities and settings of a particular WAF in order to create protection profiles. In an amicable way, working with WAF requires a separate engineer to constantly make changes to the settings.
  • He needs to constantly interact with the developers of the site and decide which vulnerabilities to close on WAF, which to fix in the code itself. Still, WAF is a countervailing measure, and some problems are best addressed at the development level.

Large companies with a large staff can cope with these three points. We wanted to take into account the requests of smaller businesses. These clients were offered a turnkey service managed by our engineers. They began to collect it from the issues of architecture.

The service was placed in a PCI DSS compliant cloud. This certificate is important for sites that work with payment data: retailers, service providers, processing companies.

The WAF service also got into the PCI DSS scope.

The service fault tolerance has been considered separately. For FortiWeb, the vendor has provided several cluster options. We tested different methods of session synchronization and settled on the Active Backup mode: one FortiWeb virtual machine remains the main one and drops part of the traffic to the second. In this case, when a node falls, the service is not available for more than 10 seconds. Configured the location of the machines so that they always go up on different hosts.

Then we solved the issue with the separation of clients in the cloud. FortiWeb does not support multi-tenancy: we cannot segment WAF into domains, as we did with our NGFW-as-a-Service based on FortiGate. What remains is the policy separation. If we leave such policies at the mercy of our clients, there is a risk of influencing our neighbors. Therefore, our IS engineers responsible for the service create client sites for WAF themselves and monitor the service settings. And for clients we provide a separate tool with reporting and statistics.

Therefore, the service itself was not limited to only WAF, we added to it:

  • Qualys vulnerability scanner,
  • DDoS protection from Qrator,
  • ELK for statistics collection, visualization and data analysis.

I’ll tell you a little more about each component.

We scan sites for vulnerabilities

To search for vulnerabilities in the code of sites, we added Qualys – a solution for scanning and analyzing the security of web applications. We already had a separate vulnerability scanning service deployed, it only remained to configure it to work together with WAF.

The first time we launch the scanner before setting up the site for WAF and send the report to the application developers. Only after that we close the vulnerabilities on WAF that cannot be eliminated at the code level.

Next, we set up a monthly scan and send a report to the client. And for the future, we are testing a scheme where the report will immediately be uploaded directly to FortiWeb.

Understanding traffic cleaning

FortiWeb protects well against application layer attacks, for example: if a web application is attacked with a large number of GET requests, DoS-Protection on WAF will help. But in addition to it, you need a separate solution for DDoS attacks below the level, for example, SYN-flood. Some clients have their own anti-DDoS, which we integrate with our WAF service. But this is rather a special case.

For other cases, the service has Qrator.Ingress – a solution for protecting infrastructure from DDoS attacks. It protects the L2 to L4 link, analyzes and cleans up traffic.

We created 10 Gigabit connections with Qrator to send already cleaned traffic to WAF over a secure channel. The service schema now looks like this:

The client’s infrastructure can live anywhere, on any hosting. After setting up a site for WAF, only traffic checked by the service is sent to this hosting. Here’s a simplified diagram of how it works:

  1. We have a “white” subnet that we advertise to the channel with Qrator.
  2. We allocate 1 new IP address to the client from this subnet.
  3. On WAF, we create Server Policy and HTTP Content routing policies for the site, add domain names there and indicate where to send them.
  4. FortiWEB terminates TLS connections on itself, so we load a chain of certificates on it.
  5. Then we ask the client to change the DNS record of the site and, for the required domain names, specify the IP address from the Qrator network in it.

What’s the bottom line? When a user launches a browser and accesses a WAF-protected resource, their DNS traffic is first routed to the cleanup centers in Qrator. Then the traffic goes to WAF via the secure channel. And only after verification, the user gets access to the client’s web server. Here’s the whole path:

But even if the site is registered for WAF, this does not mean that it is protected. You need to configure the Web Protection Profile – a set of rules and protection settings for the site. This is done separately for each client.

The situation with attacks is such that you cannot set up a profile once and you cannot forget about it. Therefore, we analyze the situation on each site and improve protection: we add new security mechanisms, adjust the profile so that there are no false positives. For the initial configuration, we use the information we received from the Qualys vulnerability scanner.

Working in a team: studying attacks with a client

Administration of the service by our engineers includes setting policies and, if necessary, manually blocking attacks. This is the responsibility of FortiWeb specialists who understand the nuances of protecting websites. But at the same time, our engineers cannot know all the logic of the client application. If we immediately block any suspicious activity, we may accidentally block a legitimate user. So we work in conjunction with the client’s specialists and develop a joint strategy to repel attacks.

Among the clients there are companies without large IT and information security departments. Outsourcers often develop applications for them, so it takes some time to fix vulnerabilities in the code. On WAF, you can offer workarounds and quickly neutralize the attacks that have arisen. In this case, we have developed regulations and prescribed when to consider the situation critical and immediately take action, and when to agree on locks with the client.

Monitoring in conjunction with ELK helps us to track critical situations. Last time we already talked about how it works technically. ELK allows us to manage security information without adding an expensive SIEM (Security information and event management) system to the service.

On monitoring, we set up alerts for immediate response. The first line of support works around the clock, after notification, immediately assesses the situation according to the regulations and acts at its own level, or transfers the incident to security engineers.

If the client himself is ready to monitor the work of WAF, we provide him with access to the analysis system based on ELK. In this solution, we set up permissions for specific indexes. The client will manage analytics and reports without influencing the WAF itself.

What is the result and how it will develop further

As a result, we have a comprehensive solution for protecting web applications. In addition to protection using WAF, we can configure balancing of web servers, caching, redirects and, thus, offload the main site.

For the future, we have provided a margin for scaling. For balancing, let’s connect FortiADC, a controller for application delivery and load balancing in high-performance systems. It supports SSL offloading of servers and improves the performance of web applications.

The service is charged based on bandwidth and is suitable for not very heavy sites. If the client has gigabits of traffic, then we offer not a cloud service, but a private solution. To make the service transparent for customers, we plan to display WAF data in their personal account.

If you are interested in knowing the details, I will be happy to answer your questions in the comments. Or register on seminar on WAF on November 26 – there will be an opportunity to ask questions not only to us, but also to the technical specialists of Qualys, Fortinet and Qrator.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *