Best tools and demos
For whom
In this article (or digest), I'll share key tools, demo applications, and open source projects that are essential for a practical understanding of WebRTC. There will be no tutorials or detailed explanations about this or that part of WebRTC, but rather a digest of resources that will help you better understand the topic. If you've been working with this technology for some time, you're unlikely to find anything new for yourself.
WebRTC Basics: What You Need to Know
If you are familiar with how video calls work in browsers, feel free to skip this section (and possibly the entire material).
Most popular video calling platforms (except Zoom) use WebRTC technology to implement their products in browsers. It is the de facto standard for creating real-time communication services and is supported by all modern browsers.
WebRTC consists of several main components:
Media Capture API: It allows you to capture audio and video streams from your device's camera and microphone.
Connectivity API: It includes protocols and methods for establishing direct connections between browsers to transfer media and real-time data. This includes exchanging network information, selecting the best route for data, and managing the connection.
Codecs and media processing: WebRTC supports various audio and video codecs for encoding and decoding media streams. Media processing tools are also available, including echo cancellation, noise reduction, and automatic gain control.
Security: WebRTC uses encryption for all transmitted data and media streams to ensure online privacy and security.
How WebRTC works:
Signaling: To start exchanging media data between peers, a signaling process is required, which is not directly included in the WebRTC specification, leaving developers the freedom to choose a signaling method (via WebSocket, REST API, etc.). Signaling is used to exchange meta-information: media type, codec parameters, network addresses and ports necessary to establish a connection.
Establishing a connection: After the exchange of initial information, ICE (Interactive Connectivity Establishment) mechanisms are used using STUN (Session Traversal Utilities for NAT) and TURN (Traversal Using Relays around NAT) servers to bypass the NAT (network address translator) and establish a direct connection between peers .
Data transfer: Once the connection is established, streaming data begins to be transferred between peers over a secure connection, using RTP (Real-time Transport Protocol) for media and SCTP (Stream Control Transmission Protocol) for data.
Adaptation and Processing: During transmission, WebRTC can adapt to changing network conditions, changing the quality of the stream to optimize performance. Media processing also occurs, including noise filtering, echo cancellation and other functions to improve call quality.
Next, in three sections, I will talk about the tools/services and demos that helped me study this technology in a practical way, and then I will share opensource products that will allow you to understand in detail and in practice how WebRTC works in large products.
Tools
Chrome Tools
The Chrome browser provides several powerful tools for analyzing WebRTC:
Webrtc-internals
Let's start with chrome://webrtc-internals. This is probably the most basic tool that one gets acquainted with at the very beginning of the journey when working with WebRTC. There you can find a lot of information about current connections, which helps with debugging and reverse engineering of other products.
Here you can find out which codecs are used by existing video calling services. For example, if you join a call in google meets and open chrome://webrtc-internals, you can see that google meets uses the AV1 video codec.
Or, for example, you can see the number of frames sent per second and their resolution
The main limitation is that the data is only available during a call (or if you enable recording in advance), which means that the tool cannot be used during unexpected problems.
More information can be found here: https://getstream.io/blog/debugging-webrtc-calls/
Webrtc-logs
This section is used much less frequently as it contains more detailed information and is necessary for a more detailed analysis of complex problems. Unlike chrome://webrtc-internals, it contains logs in text format about previous sessions. This way you can view logs for an already completed call, chrome://webrtc-internals cannot do this.
Inside you can find information about the start and end of video calls, media stream parameters, codecs, video resolution, frame rate and other media stream quality metrics.
Some of the information from these logs can be used to build metrics similar to those in chrome://webrtc-internals. Unfortunately, Chrome does not provide this functionality, but there is an open source product that helps with this task: https://fippo.github.io/dump-webrtc-event-log/
This tool is ideal if there is a problem on one of the live calls and when real-time analysis is not available. These default logs are saved to the directory …/Google/Chrome/Default/WebRTC Logs/
TestRTC
analyzeRTC from the testRTC service is in something of a beta testing. The entire platform is paid, but you can analyze your dump completely free of charge. In general, nothing new can be found here after chrome://webrtc-internals, since they have the same data source. But the display of this data in analyzeRTC is much better.
For example, information about a video stream is presented particularly conveniently:
You can also see the overall rating of the call and audio quality here:
Score is the overall score of the script from 0 to 10, taking into account audio and video across all channels. It helps evaluate the performance of a service and requires subjective interpretation based on experience with the application. MOS (Mean Opinion Score) evaluates only the audio channels of a scenario, presenting subjective audio quality on a scale of 1 to 5, with above 3 considered good quality, 2 to 3 fair, and below 2 poor.
A short video tutorial can be found Here
Demo Applications
Official Demos
The following are official examples of using WebRTC. In this case, this is not a tool, as I described above, but a set of working examples. These examples cover a wide range of functionality, from basic connection establishment between peers to more complex scenarios such as data exchange and media stream management.
Each link contains a working demo and a link to the source code on GitHub. Using only these examples, you can already understand how to develop a rather complex web application with video calling functionality. In general, these examples help to better understand each aspect of WebRTC in practice in order to better consolidate theoretical knowledge.
SDP Explanation
Next, let's talk about SDP – the protocol used to establish a connection before starting a call. The format itself is described in RFC 8866it is designed to agree on media stream parameters before a call, such as codec formats, video resolution and media types (audio, video, data) needed to establish a connection and exchange media data.
The protocol is designed in such a way that, on the one hand, it is human-readable, and on the other, it takes up as little memory as possible, as a result of which it is quite difficult to immediately understand what is contained in a particular message. It is to solve this problem that you can use this tool.
In interactive mode, you can read about each line of the SDP protocol, saving time searching for explanations in the RFC.
Media test
In addition to establishing a connection and transmitting media streams, an important component of any video calling application is the processing of these same media streams. This tool allows you to understand how video can be processed.
Here you can experiment with various aspects of video streams, such as creating a stream from scratch or from a camera, choosing video resolution and frame rate. You can also apply transformations to frames, such as changing colors, converting to black and white, encoding/decoding to H.264, and adding overlays to track display time in the
After experimenting, you can see the processing results, including processing time at different stages (such as converting to RGBX, adding backgrounds and overlays), frame display time, total time from start to finish of processing, and queue time. Each indicator includes the number of frames processed, the average, median, minimum and maximum processing time. This data helps you evaluate the performance and efficiency of video processing in the application and understand how much each operation “costs”.
Slides from the authors.
Simulcast Playgrounds
First
Another demo, this time unofficial, but still open source. Here we focus on Simulcast. This is an approach that allows conference participants with different network bandwidths to receive video from you in different resolutions (the higher the bandwidth, the better the resolution).
In order not to use server resources, each participant sends several video streams in different resolutions. It is this algorithm that is simulated in this example. It allows you to customize video transmission settings, including selecting resolutions, frame rates, codecs, and Simulcast-specific settings such as adding and removing layers, changing priorities, and setting encoder parameters.
If you've never heard of Simulcast, it's best to spend a little time learning the theory before jumping into this demo. I would recommend looking at this material on Habré:
https://habr.com/ru/companies/flashphoner/articles/564714/ (Search for the keyword 'simulcast' if you don't want to read the whole article)
Second
Another variation of the simulcast demo, not at all flexible, but much more representative. You can see the default configuration with three parallel threads and metrics from chrome://webrtc-internals. It looks more familiar and does not distract with additional settings. An excellent addition to the tool described above.
Pagination Demo App
An excellent tool for understanding the display of incoming video streams and managing the resources they use. Unfortunately, the deployed demo did not work at the time of writing, but it would not have been of much help in any case, since the main value of this product is its source code.
There you can figure out how to properly manage incoming video/audio streams from other call participants. This is especially important when there are a large number of participants, when all of them do not fit on the screen and you need to add other pages. There is no point in accepting a stream of data from participants located on other pages, and this needs to be handled correctly.
The most interesting component is 'components/PaginatedGrid.js', it contains almost all the logic for managing rendering and flow control. A description of the source code and the entire application in general can be found in official documentation.
Opensource products
Jitsi Meet
One of the most popular open source products for development related to video calls in the browser. In fact, jitsi contains not only the web client, but also all the other necessary parts to build the entire product. This is a great resource for diving very deep into the topic of RTC, but it will take a lot of time as jitsi is very flexible and quite an old product with a huge code base.
This tool in general has a fairly high barrier to entry due to its size and age. The documentation is present, and quite detailed, but again very large: https://jitsi.github.io/handbook/docs/category/developer-guide
Janus
Janus Gateway is an open source WebRTC server that solves a similar problem. But unlike Jitsi Meet, Janus focuses on providing a low-level API for managing media streams, making it more flexible but potentially more difficult to use for newbies. This product is also more suitable for those who are interested in the server part rather than the client part, with which Jitsi Meet can better help.
Very detailed information is available for Janus documentation and a whole bunch of separate demo (which, by the way, were not included in the previous section, but I would still recommend reading them).
Mediasoup
MediaSoup is another WebRTC server. Unlike Jitsi Meet and Janus Gateway, It solves the same problems as Janus Gateway with the difference that mediasoup has both a server and a client part.
Other products
In fact, there are a whole bunch of opensource products for building video calls, I indicated those that I consider the most useful and convenient when studying the technologies and approaches used in WebRTC.
You can find out about other products at link (in English).
Instead of conclusions
There were no arguments or innovations in the article, which means that no conclusions can be drawn. I just described all the tools that I used, I am sure that this is not a complete list. I would appreciate your recommendations for tools and services that I missed in the comments!
![[Личный опыт] Is there life in South Africa and is it worth moving there for an engineer](https://prog.world/wp-content/uploads/2021/03/kk5eqgfafdgxbjuotowb6yv0lcm.jpeg)
