we analyze Habr, Habr Career and Stepik

It turned out to be not so easy to mark up Stepik courses. After visually evaluating the model’s predictions, I came to the conclusion that it performed poorly here. Only about 200 courses out of 1.5 thousand were marked up correctly, so I had to go back to my ancestors, namely ~~mining fire~~ labeling data manually. At some point, I got tired (after all, this is not how I imagined the life of a programmer!), So having marked a third of the data, I trained the neural network I had already written on them, and for the rest I got predictions.

Analysis

As a result, I had thousands of observations from different sites ready for analysis in the period from 2016 to 2021. For all datasets in each direction, I built trend lines, and in addition to the number of observations, I also investigated the interaction of people with these areas, that is, for example, the number of comments and views. It is worth noting that I considered percentage ratios, not absolute values - this way you can avoid the influence of the growing popularity of one of the three selected sites. In addition, we managed to collect interesting statistics on the most popular professions from the career.habr website and analyze trends in them.

What do we have?

Perhaps, I will not talk in detail about the trends in all the selected areas. I will focus only on “technical” ones (and if you are interested, I can tell you about the rest in the comments).

The strongest growth is observed in the direction “Iron and Electronics”: the number of vacancies is steadily growing, as is the interest of people:

The percentage of vacancies in the direction of “Iron and electronics” among all areas. Orange line – values, blue – trend line

Percentage of articles in the direction of “Hardware and electronics” among all articles. Orange line – values, blue – trend line

The situation is somewhat sadder with web development: the number of vacancies and articles is decreasing, while user activity (comments, saves, views) is growing:

Percentage of articles in the category “Web development” among all articles. Orange line – values, blue – trend line

This is also noticeable when analyzing the demand for Frontend developers and PHP developers – the number of vacancies is falling:

The percentage of vacancies for the position of Frontend developer among other vacancies. Orange line - values, blue - trend line — The percentage of vacancies for the position of Frontend developer among other vacancies. Orange line – values, blue – trend line

The percentage of vacancies for the position of PHP developer. Orange line - values, blue - trend line — The percentage of vacancies for the position of PHP developer. Orange line – values, blue – trend line

The reverse situation with machine learning: demand in the labor market and in the educational sector is gaining momentum (albeit not so much), while the activity of users on this topic begins to fade:

The percentage of views of articles in the direction of “Machine learning”. Orange line – values, blue – trend line

Percentage of courses in Machine Learning among all courses. Orange line - values, blue - trend line — Percentage of courses in Machine Learning among all courses. Orange line – values, blue – trend line

Percentage of comments on articles in the Machine Learning category among all comments. Orange line - values, blue - trend line — Percentage of comments on articles in the Machine Learning category among all comments. Orange line – values, blue – trend line

The same conclusion can be drawn about administration and Devops: this direction is becoming more and more popular with employers, although the interest of people is slightly declining:

The percentage of vacancies in the direction “Administration and DevOps” among all vacancies. Orange line – values, blue – trend line

Percentage of articles in the direction “Administration and DevOps” among all articles. Orange line – values, blue – trend line

The percentage of vacancies for the DevOps Engineer position among all vacancies. Orange line - values, blue - trend line — The percentage of vacancies for the DevOps Engineer position among all vacancies. Orange line – values, blue – trend line

A similar situation with information security: The interest of employers in this area is growing, which is reflected in the increasing demand for courses, but among blogs, the popularity of this topic is greatly reduced.

Percentage of participants enrolled in Information Security courses among all participants. Orange line - values, blue - trend line — Percentage of participants enrolled in Information Security courses among all participants. Orange line – values, blue – trend line

The percentage of articles in the direction of “Information security”. Orange line – values, blue – trend line

Despite the decreasing number of articles in the direction design and testing, in general, interest both from blog readers and course participants, and from the labor market, is slightly increasing.