New PVS-Studio Features for Notifying Developers about Found Errors

PVS-Studio support often receives suggestions from users to improve the product. We are happy to implement many of them. One of the last such proposals was related to the refinement of the utility for automatic notification of developers (Blame Notifier). We were asked to teach her how to extract the date / revision of the code, for which the analyzer issued a message, using blame information from the version control system. This refinement allowed us to expand the capabilities of the utility, which we will talk about in this article.

How it all started

One of our customers wrote to support that he needs to distinguish between the analyzer’s warnings and the ones found earlier. In the correspondence, it became clear that he needed to focus on warnings for new or just changed code. We hastened to inform you that it is possible. to carry out with help SonarQube in conjunction with our plugin. The client’s response was not long in coming. He noticed that using a third-party and rather massive tool, such as SonarQube, is impractical for these purposes.

Since the client has a large code base, we proposed to logically divide the use of a static analyzer into before and after the implementation of static analysis, temporarily suppressing previously found warnings using our mechanism suppression. After that, the analyzer would issue warnings only for new or changed code. And in addition to this, they talked about the implementation of Blame Notifier to immediately notify all involved persons about new warnings. The client was interested in the offer, but due to the fact that he wanted to see both old and new warnings in the report, this mode of operation seemed not very convenient to the user. In addition, he gave feedback on Blame Notifier: “It would be cool if the warnings in the notifications were more informative, for example, add the date and revision number.”

And then it dawned on us. Previously, we thought that for smarter and more informative alerts, you need to create your own database. Based on it, we could track the life cycle of each alert and, accordingly, determine which of them are new. But after the client’s remarks, we came to the conclusion that the version control system is the very database from which you can get all the necessary relevant information. Indeed, by linking to the commit revision numbers, we can calculate their linking to the modification time of the corresponding lines of code. Moreover, we already get this information through blame – all that remains is to use it.

We offered the client the following option:

you will receive the resulting analyzer report with all warnings;
Blame Notifier can be configured so that only the newly changed code gets warnings in the HTML report. In addition, the warnings will contain the date and revision number.

On this we agreed!

About the utility

The purpose of Blame Notifier is to automate the process of notifying developers whose code was issued a warning by PVS-Studio, for example, after nightly builds. The utility allows you to generate an HTML report both for a specific developer with only his warnings, and for a super-user who receives a full report with all warnings. The full report is by default grouped warnings by developer, which, in turn, are sorted alphabetically. This functionality is extremely useful, as it will immediately signal about new analyzer triggers to all interested parties. As you might guess from the name of this utility, it works on the basis of blame information obtained for files checked by the analyzer from the user’s version control system.

If you are unfamiliar with this utility, you can learn more about it in this article…

What’s new

In addition to the developer’s name, the blame information now extracts the date and revision of the last change in the code that PVS-Studio is cursing at. Additional information retrieved allowed us to add new options to the utility:

–sortByDate (-S) – allows you to generate an HTML report with warnings sorted by the date the source code was modified, due to which the analyzer warning was issued. Alerts for a specific date are grouped, in turn, by developer.
–days (-d) – the HTML report contains warnings for the code, the modification date of which is less than N days from the date of the current launch of the utility.

Note… Date / revision extraction is supported for the following version control systems: SVN, Git, and Mercurial.

The default format for the utility’s HTML report is as follows:

New HTML report sorted by date:

How it can be applied

New options allow you to simulate the behavior of SonarQube when loading an analyzer report using ‘sonar-scanner’. More about this.

Note. If you are already using PVS-Studio in conjunction with SonarQube, then the application described below will be irrelevant for you, since the corresponding processing of new warnings is already built into SonarQube. But if you are using PVS-Studio separately and there is no possibility / desire to use SonarQube, then this may interest you.

To begin with, SonarQube uses a code quality approach called ‘Clean as You Code’. Its essence is that developers should pay increased attention to the reliability and security of new code that has just been added or changed. Old code that has been proven to work in ‘production’ over the years should be sidelined and focused on what is happening “today”, thereby preventing new problems from arising. And to periodically return to existing problems and correct them. You can learn more about this approach in blog SonarQube developers.

This approach is implemented as follows. The main page of the project has a dedicated area with new issues for a configurable amount of time. For new problems, a ‘Quality Gate’ is configured, the so-called indicator of compliance of the new code with the specified threshold metrics. For example:

the number of new bugs;
number of new vulnerabilities;
technical debt ratio;
coverage of new code with tests;
… other.

For example, every morning a fresh analyzer report will be downloaded, and if the threshold value of any metric is exceeded, then the ‘Quality Gate’ will signal this to you.

The visual representation of the ‘Clean as You Code’ approach is intuitive. Here’s an example of what it looks like in SonarQube 7.9.4:

SonarQube is a professional tool that provides functionality that allows you to conveniently and effectively control the quality of a software product. For example:

breaks down problems in the code into existing and recent ones;
provides various metrics and graphs;
allows you to filter problems by criteria;
allows you to watch warnings found by code quality control tools directly in the code being checked right from the web browser;
… and much more.

If SonarQube is already in use, then in order to integrate the results of the PVS-Studio analyzer with it, you need to familiarize yourself with article…

But what if you are not using SonarQube yet?

If you have the opportunity, you can install it, but for this you must first carry out the procedure of deployment, installation and initial configuration of SonarQube, which may well cause certain difficulties. For example, when:

starting the SonarQube server;
setting ‘Quality Profiles’;
setting ‘Quality Gates’;
using ‘sonar-scanner’;
… etc.

However, not every development team needs all the features SonarQube provides. And therefore, a long and rather labor-intensive procedure for deploying and maintaining the operation of this system may well turn out to be redundant.

And this is where our Blame Notifier utility comes to the rescue. It has already been able to replace some of SonarQube’s capabilities in the area of developer notification before. With the new capabilities of the utility, it is now possible to simulate a light version of the ‘Clean as You Code’ approach, where the main metric of code quality will be the appearance of new analyzer warnings. In this mode, the mailing list will contain warnings for the code, the modification date of which is less than N days from the date of the current launch of the utility.

Let’s repeat the SonarQube behavior. To do this, we specify 10 days for the ‘-d’ option and sort the warnings (-S) by the date the code that caused the warning was modified. In this case, the HTML report will be as follows:

As you can see, the report contains 11 new warnings for the last 10 days. It looks almost the same as in SonarQube, but it is much easier to get such reports.

This “lightweight” approach, of course, has its drawbacks:

the quality of the new code is assessed by one metric;
triggers on code whose modification date is outside the considered period will not be included in the HTML report. To do this, you need to get a full report for the entire time using an additional launch of ‘blame-notifier’ without limiting options.
and, of course, there is no way to navigate the code being checked directly using a web browser

It is also worth mentioning that the use of the Blame Notifier utility and the PVS-Studio plug-in for SonarQube is available only in the PVS-Studio Enterprise license.

Conclusion

The introduction of static analysis technology into development implies not only launching the analyzer and receiving a report. Usually hand in hand with a static analyzer are auxiliary utilities / plugins, the use of which makes it easy and convenient to maintain the quality of your code.

For example, PVS-Studio has many plugins (for Visual Studio, SonarQube, Jenkins, Gradle / Maven / IntelliJ IDEA), a utility for more convenient conversion of reports PlogConverter, a developer notification utility blame-notifier… The PVS-Studio team, relying on its experience and user feedback, is constantly improving its product. In order not to miss all the changes and keep up with the tool, do not forget to keep an eye on blog on the official site.

The new Blame Notifier operating mode allows you to “lightly” perform some of the functionality that was previously available in our product only in conjunction with SonarQube. However, I would like to repeat once again – we in no way urge you to abandon the use of SonarQube. The new mode makes it much easier to try this approach to working with static analysis. We believe that this mode will be relevant for many teams.

If you have any suggestions for improving our product, do not hesitate to put it off and immediately write us.

If you want to share this article with an English-speaking audience, please use the translation link: Maxim Stefanov. PVS-Studio New Features for Notifying Developers About Errors Found.