DevTools Strike Back

Disclaimer

This article was born after my participation in organizing several tracks about tools for developers at major Russian industrial conferences in 2024. Its main goal is to talk about what is interesting to the development community, share your understanding and throw in some links “to watch”. Therefore, for a full immersion recommended follow the links and watch a couple of videos. Nuff said.

DevTools in each of us

Themes around developer tools always generate interest and discussions among programmers, and especially in companies whose core business is entirely related to software development. And most importantly, these discussions never end. no stabbing and leave no one indifferent. I am sure that in the comments under this article, holy wars of witnesses should begin stolen IDEs or sovereign githubs.

It is clear why this happens – after all, the mysterious category of DevTools includes absolutely everything that developers use during their daily routine: from IDEs to code analyzers, from profilers to debuggers, from code hosting platforms to CI/CD, issue trackers and many other tools. And each developer has their own opinion, pain and experience associated with them.

That is why at the last festival Positive Hack Days 2024 in Luzhniki we did not fail to take advantage of the moment and arranged such clickbait. After all, in Russia this topic has obviously become especially hot (for example, after a month and a half GigaConf site from Sber we already collected a similar track about DevTools). At PHDays I cheated a little and expanded the topic to an even more general one, collecting the track “Languages and Tools”. That is, we discussed not only standard tools, but also the most important ones. DevTools for any developer: programming languages ​​themselves, compilers, interpreters and related technologies. As a result, unexpectedly for me, we spontaneously got a continuous narrative of seven reports with cross-references to each other and very lively communication.

As a result, we have the track opened Chairman of the EuroLLVM community and contributor to LLVM/Clang Andrey Bokhanko, who has been personally involved in the development of these projects for more than ten years (by the way, I recommend his article). And the cherry on the cake was report from a Java contributor and the author of the Dual-Pivot Quicksort in the JDK Volodya Yaroslavsky. And of course, when the conversation turns to tools for developers, it couldn't do without the authors of popular open source libraries, who came out with a hardcore pro fuzzerssampling profilersconcepts compilers (using MOJO as an example) and Kotlin multiverse.

Who are these DevTools of yours?

And just at PHDays, in between hardcore talks, I managed to do easy report “Open source developer tools that We deserved”. In it, I gave a small, superficial overview of development tools and processes that are useful when creating your own projects. By and large, this report answered the questions: what are DevTools? What is generally available to a developer, and in what order should priorities be set in order to build your project from scratch?

Speaking about developer tools, it seems that one can immediately define some info-gypsy pyramid of “Needs”, which always determines the sequence in which almost any project grows (we once described such a pyramid for Java developers on Jpoint). This movement usually occurs progressively from a raw project, in which a programming language is simply chosen, to a state in which automation of development is brought to absolute. (like the Sith) right up to auto-refactoring of code, all sorts of conveniences in CI, code checks and even development of projects by autonomous AI assistants like AutoGPT (by the way, a great list of all such AI tools collected here). In this article we will try to look at the most interesting and important of which in the context of writing code by an individual developer.

What does Open-source have to do with it?

Open-source itself is the flagship of DevTools, because it is here that the community is so large and active that most ideas for such tools appear and are immediately tested here. And the leading role in the development of this community continues to be played by GitHub, which provides a platform for all kinds of plugins and actively explores how to improve the software development experience. The entire industry movement is tested here. Therefore, I hope that I will not offend the creators of sovereign Git hosting, but my examples will be based on what is happening on this platform. By the way, many developers do not know that you can get acquainted with new trends that the Github research team is currently focusing on at githubnext.comThe authors, without any embarrassment, write on their website: “Exploring the future of software development.”

But, by and large, all modern research is currently being conducted around AI assistants. From unrelated to Copilot and close to me we can only highlight the movement towards collaborative development and just a little bit about CodeQl:

• Realtime GitHub And Collaborative Workspaces: the ability to collaborate on a project in real time, like in GoogleDocs. In the first case, we are talking about modifying documents and code directly on the platform on the web, and in the second case – in the IDE.

• Incremental CodeQL: a study that is close to me personally, as a person related to code analysis. Increasing the productivity of code analysis in CI due to the fact that when the code base changes, the construction of the CodeQL analogue of the Code Property Graph occurs incrementally without the need to rebuild the entire graph from scratch.

  By the way, if you suddenly want to dive into the wonderful world of CodeQL, CPG analysis or incremental code navigation, I recommend three talks: “Incremental, zero-config Code Nav using stack graphs” from Douglas Creager, “Code Analysis and Graph Databases” from Andrey Shcheglov and “Java Code Analysis with Database and Domain Specific Language” by Pan Linjie.

Github Next still really grabs me and sometimes makes me sad – this is sunset modelwhen the sun sets on Microsoft's economic feasibility study. For example, my favorite project of motley repository customization died GitHub Blocks. I hope at least that one day sovereign Git hostings will finally complete this functionality.

In general, GitHub, despite its name, has long ceased to be an ordinary Git hosting service and has become a full-fledged development environment, with its own Cloud IDE (due to Codespaces), with its CI (due to the extremely convenient Actions) and even the ability to write your own plugins that can be natively integrated into the review process. In the developer pyramid of needs, the GitHub eco-system covers almost all the steps, except for the full-fledged role as a Cloud provider.

And even with that, GitHub doesn't stop and turns into some kind of social network for developers with Stackoverflow functionality (Discussions), achievement mechanisms, moderately vain ratings and even their own “rock stars” (stars.github.com). Moreover, entire startups are built around this rating system, like opensource-heroes.com or gitstar-ranking.com which collect and rank people by stars on projects.

Market of development platforms and IDEs

However, in this beautiful story with a platform that covers all the needs of the developer, there is a fly in the ointment: to complete a simple quest from “I wrote the code in the IDE” before “I put the code in open-source” is becoming much more difficult due to political circumstances. Microsoft has already applied restrictive policies to developer groups, forcing more and more sovereign platforms to emerge that compete with GitHub. And this is where the really strong players emerge, such as Gitee (the largest Git hosting in China from OSChina), which are starting to eat away at Github's market share.

Well, it is also obvious that there is also some movement towards decentralized platforms for developers and Git hosting. For example, such as the project radicle.xyz.

My friends from JB will probably criticize me, but the same story is now visible with IDEs, which are being developed in one way or another by many large companies that are trying to cover their own needs and repeat the success of the two market leaders, who produce both desktop IDEs and cloud solutions. Microsoft is represented by its VSCode And Codespacesand JetBrains with its own set of IDEs and collaborative development tools: plugin CodeWithMe And Space.

It is also worth noting the struggle between these two companies: judging by large-scale surveys such as StackOverflow SurveyVSCode is the leader in the broad market, although the Intellij platform still completely dominates the JVM language market. This is because Microsoft's VSCode has very weak support for Java (via a plugin from Red Hat).

Continuous Integration

In our hierarchy, it is also worth highlighting systems for Continuous Integration or CIwithout which many tools for building, testing, automatic validation and code analysis would not have been actively developed. After all, it is CI that allows you to regularly and automatically run all these checks and develop the right patterns of approaches to development in the development team. CI covers two main needs of the developer: building for each PR and commit to the main branch, as well as regular automatic testing.

If we exclude team management tools from the survey The State of Developer Ecosystem 2023 (from JetBrains), and we leave only the tools on which the development of programs directly depends, then it is CI that will take the leading position after collaborative development platforms.

In the same survey, unexpectedly for me, Jenkins has the leading position in the CI category in terms of number of uses, even surpassing the very simple and convenient platform integrated into GitHub Actions And GitLab CI.

On CI's Shoulders: What Makes Your Project Better

But CI and its functionality are definitely not just the basic minimum we listed above, with the project build. These are also nice bonuses:

• Examination code base by code analyzers. By the way, Here Great list of popular open source code analyzers.

• Automatic update code base. From project versions in the readme to automatic code refactoring.

• Collection metrics each code change: from test coverage percentage through CodeCovto code metrics, for example Cognitive Complexity. Well, and collecting general metrics for the project (OpenSSF Best Practices, Codebeat and many others).

• Well, the main convenience is helping developers c code review (Code, Code Climate, CodeFactor). It's interesting that GitHub had big study regarding automatic code review from Copilotwhich was completed in December 2023 without fully adding this functionality to Github.

Therefore, large platforms with built-in CI (such as Github) provide the ability not only to conveniently create and host such plugins for workflow, but also to natively integrate them into the review process due to a special format SARIFthe tooling for which allows you to upload static analysis reports directly to Git hosting and display these results directly during the review.

This is especially applicable to applications that analyze code, since the analyzer's advice appears as comments. a living person directly to the review. At the same time, there are two very large aggregators of code analyzers in open source that use these formats: Spotless And Super-linterwhich are worth paying attention to.

It is also worth mentioning separately that scanning projects for problems is fashionable among large companies. third-party dependenciesand we are not only talking about supply chain attacksbut also about the banal use of code or dependencies that could potentially have an incompatible or strict license. Among such utilities, it is worth highlighting the platform FOSSA.comwhich performs a comprehensive check of project dependencies. And of course Dependablebotwhich became part of GitHub and performs a comprehensive check for vulnerabilities in dependencies. And also RenovateBotwhich automatically allows you to update project dependencies to new versions on a scheduled basis.

In conclusion

We can talk endlessly about developer tools, because only on Github Marketplace You can find thousands of small utilities that help a developer do his main job – write code.

And so in this article we talked only about the basic things and principles that are worth paying attention to. I really hope that I will have enough strength to give birth to a series of more in-depth articles on this topic. In any case, I am sure that the Dev tools industry will continue to grow by leaps and bounds, especially with the active help of artificial intelligence. And more and more tools will be to strain and hit the head with alerts make life easier for the average developer. And the developers will continue to fight back.