sources not out of the box
In the previous article we talked about how you can connect standard sources to Wazuh, which come, as they say, “out of the box”. With them, everything is relatively simple: we follow the steps presented in the instructions for connecting Wazuh on the source (for example, Syslog redirection), make the necessary changes on the agent side and everything should start working without problems.
However, in reality everything is not so simple. Many organizations have home-written or highly customized software, the logs from which SIEM cannot normalize. That is, in fact, he is simply unfamiliar with this type of sources. If such a non-standard source transmits events via Syslog, then in SIEM, most likely, we will see the event almost in its raw form. That is, the fields that are usually filled in during normalization (Source/Dest IP, Username, Hostname, etc.) will not be filled in in this case.
In this article we will talk about what can be done using Wazuh to normalize events from such sources. In fact, finding an example of logs that SIEM cannot normalize out of the box is not so easy. As a rule, messages from the OS and main services are normalized without problems. For this reason, when training one commercial SIEM, it is proposed to parse logs from a coffee machine as an example of logs for normalization.
Regular is our everything
Regular Expression Language (RegEx) is a formal language used in text-based computer programs to search for and manipulate substrings in text, based on the use of metacharacters. For the search, a sample string is used, consisting of characters and metacharacters and defining the search rule. To manipulate text, a replacement string is additionally specified, which can also contain special characters.
When analyzing raw logs, we need to use the regular expression language in order to extract the values of the required fields from raw events. In this article, we will not consider working with RegEx in detail, since more than one article or book is devoted to this topic.
Here I will just note that without understanding the basic principles of how regular expressions work, it will be extremely difficult to write and debug your own normalization rule.
Writing a new rule
In order to write our own normalization rule for a particular event, we first need to have an example of the event itself.
So in the example below we see a login event from the MyHost node and the example process:
Dec 5 22:45:01 MyHost example[12345]: User 'admin' logged from '192.168.1.10'
In fact, it is desirable to have multiple event records of the same type as examples. Sometimes you may encounter the fact that the same event in different cases may or may not contain certain data.
To work with the rules, we need the files /var/ossec/etc/decoders/local_decoder.xml and /var/ossec/etc/rules/local_rules.xml. We recommend creating new decoder and rules files for larger changes.
Each rule in Wazuh has its own number. For custom rules, use ID numbers from 100000 to 120000.
Let's return to our example log, corresponding to the program called example:
Dec 5 22:45:01 MyHost example[12345]: User 'admin' logged from '192.168.1.10'
Let's add the following entries to the /var/ossec/etc/decoders/local_decoder.xml file to normalize the logs:
<decoder name="example">
<program_name>^example</program_name>
</decoder>
<decoder name="example">
<parent>example</parent>
<regex>User '(\w+)' logged from '(\d+.\d+.\d+.\d+)'</regex>
<order>user, srcip</order>
</decoder> Here we have defined how to extract records from the raw event that correspond to our example application. And then, using regular expressions, we extract from the raw event the user name '(\w+)' and the IP address of the node from which the connection was made '(\d+.\d+.\d+.\d+)'. Here RegEx geeks can dig down Please note that this expression will not quite correctly determine the IP address template. It will respond to any numbers with three dots. For example, at 995.301.726.553, although as we all know, an IP address cannot consist of such octets.
So this expression can be improved if desired. But we will assume that no one will deliberately submit incorrect data to the SIEM input.
Next, add an entry to the /var/ossec/etc/rules/local_rules.xml file.
<group name="custom_rules_example,">
<rule id="100010" level="0">
<program_name>example</program_name>
<description>User logged</description>
</rule>
</group>That's all for the normalization rules. Let's move on to debugging.
Debugging rules
We wrote a raw event handler; you can, of course, immediately restart wazuh-manager and then see what happened. But there is a non-zero probability that Manager will not start upon restart due to syntax errors. And besides, if the event is not parsed correctly, it is not entirely clear where to look for the error or how to actually debug the rule.
To debug rules, Wazuh includes the wazuh-logtest utility, designed specifically for checking normalization results. Also, using this utility you can debug correlation rules, but we will talk about this in the following articles. So, to launch the utility, run the following command:
sudo /var/ossec/bin/wazuh-logtest
Next, we simply pass examples of raw events interactively and look at the parsing results:
Dec 25 20:45:02 MyHost example[12345]: User 'admin' logged from '192.168.1.100'
Thus, to test your normalization rules with wazuh-logtest, you only need to save your changes to the decoder and rule files. When you are sure that raw events are parsed correctly, and as mentioned earlier, it is better to use several examples of the same event, you need to restart the wazuh-manager service so that the rules start working in the system itself.
To do this, let's do:
systemctl restart wazuh-manager
As a result, we can observe normalized events in the Wazuh console.
Conclusion
Developing your own normalization rules is a challenging and very exciting process. And here, understanding all the nuances comes only with experience. Therefore, I recommend that you practice writing your own rules to understand the general principles of normalization in Wazuh.
You can get more practical skills and information security tools as part of practical online courses from industry experts.
