WebRTC through Kurento: Testing and Implementation Experience

In this article, I will share my experience with WebRTC technology and the Kurento media server during the testing and implementation phase. I’ll tell you what problems I encountered and how I solved them. I will not talk about how to develop an application from scratch, but I will give a lot of useful links. I am sure that my story will be useful to those who are going to work with WebRTC.

Introduction

The Medical Information System (MIS), which is being developed by our company, has already grown to a huge enterprise project with many microservices, messaging buses, mobile clients and so on. Some parts of the system have to be given for development and support to third-party organizations, since they are not our profile.

The Telemedicine service is one of such MIS modules. There was no experience in developing video conferencing and using WebRTC and the order was delegated. But after some time, due to various circumstances, this company stopped supporting video conferencing. Without support, this service was disabled and “gathering dust” in the repository.

And now it's time to revive this micro service. It was decided to try to restart Telemedicine on their own. Our company has grown, more specialists have appeared – it is possible and necessary to master new topics for development. I hadn’t been involved in video transmission before, but it was very interesting to understand and study such a promising technology as WebRTC.

Here are some very useful links on WebRTC technology and the Kurento server that helped me at the start:

  • Java, Spring, Kurento and Media Services
  • WebRTC: how two browsers agree on voice and video calls
  • We develop a video chat between the browser and the mobile application

Beginning of work

The task was simple: to restore the existing system of video conferencing, to take an inventory of what has already been done before and, if necessary, modify it according to the wishes of users. The first tests on virtual machines and real computers were successful. But the deployment of the system at the client brought a lot of trouble.

Let me remind you that the client already has a medical information system (MIS) covering a huge number of tasks: from the electronic queue, doctor’s workplace, document management and PACS to the medical equipment management subsystem.

When it became necessary to develop video conferencing functionality for connecting medical personnel of diagnostic centers (hereinafter referred to as DC) with remote patients, two prerequisites were set:

All conferences should be recorded and stored on the server.

Patients should not install any additional programs on their devices, except for the browser, which in most cases is already preinstalled.

WebRTC works from a browser without additional programs or plugins. And Kurento can record everything that passes through it. And besides, this media server has a good set of ready-made libraries for working with its API through Java and JavaScript, which greatly simplifies development.

The development of the server side, or rather its basis, even before I set about the task, was outsourced by the client to a third-party company. So there was a “Managing server” (CSS) – a ready-made server base, which I got for implementation.

The general idea of ​​interaction initially looked like this:

But in the process of further work, the whole system has changed and become more complicated.

First resuscitation experience

After deployment in a test network on virtual machines and on several “live” computers, many tests and experiments were carried out – everything worked perfectly. Accordingly, the time has come to introduce it into a real, working network.

For testing, a responsible victim in the form of a doctor and his workplace was assigned to help me. And the second call through the telemedicine microservice crashed!

It’s good that this happened during the beta test, and besides me and a doctor who was pleased with the adventures, no one saw this.

What is happening and why the connection is not being established was very difficult to understand. WebRTC shows no failure – it just waits for a signal to appear. Unknowingly, it was very difficult to debug somehow: the server part is working well, Kurento is silent in the logs, clients are waiting for the stream, but nothing happens.

Habr helped (praise be to him):

  • How to debug WebRTC.
  • 5 mistakes when developing WebRTC calls from the browser
  • Experience using WebRTC. Yandex lecture

It is unfortunate that I did not know these tools before.

After analyzing the log data and observing the status of connections, it became clear that both the server side and client scripts lack reaction to events in the WebRTC system. And where to get these events?

The kurento server developers provide a very convenient JavaScript library for working with WebRTC: kurento-utils.js.

For a quick start, just create an object:

new kurentoUtils.WebRtcPeer.WebRtcPeerRecvonly (options, callback ());

And in order to access events, you need to redefine the internal methods of the library. I simplified the code as much as possible to make it clearer:



// Set parameters for WebRtcPeerRecvonly
var options = {
    // generate a class with internally overridden methods
    peerConnection: getRTCPeerConnection (videoId),
    remoteVideo: videoElement, // identifier
    // method for ICE candidates
    onicecandidate: function (candidate) {
         onIceCandidate (candidate, videoId, true);
    },
    onerror: onError, // method for errors
    mediaConstraints: {
        // settings
        video: true
        audio: true}
};

// Create WebRTC
incomeWebRtc[videoId] = new kurentoUtils.WebRtcPeer.WebRtcPeerRecvonly (
    options,
    function (error) {
        if (error) {
   return console.error (error);
        }
        this.generateOffer (
        function (error, sdpOffer) {
            // Processing the result
        });
});

// We make our way to feedback methods
function getRTCPeerConnection (videoId) {
    var configuration = {
        "iceServers": [
            {"url": "stun:" + stunServer},
            {"url": "turn:" + turnServer, credential: turnCredential, username: turnUsername}
        ]
    };

    var mypc = new RTCPeerConnection (configuration);

    // Override the methods for handling events
    mypc.oniceconnectionstatechange = function () {
        state = this.iceConnectionState;
        console.info ("### iceConnectionState" + videoId + ">" + this.iceConnectionState);
    };

    mypc.onsignalingstatechange = function () {
        state = this.signalingState || this.readyState;
        console.info ("### SignalingState" + videoId + ">" + state);
    };

    mypc.onconnectionstatechange = function () {
        console.info ("### ConnectionState" + videoId + ">" + this.connectionState);
    };

return mypc;
}

Speaking of certificates

Since the article is about my experience, I will share information that modern browsers have become very strict about monitoring security. If the resource has a self-signed certificate, then even with special permission, the browser prohibits access to computer peripherals.

You can create a certificate on free resources on the Internet and configure a local network for its use, or you can download Firefox version 65 or higher. In this version, just click the button, which I agree with the risks of self-signed certificates and access cameras and microphones.

This way seemed easier to me.

Second test (already careful)

It seemed to me that the doctor would run for popcorn when he saw me in the next test. He clearly enjoyed watching me struggle with modern technology.

In fact, this system update was not a release, because I did not fix anything, I did not even know the cause of the problems. I repeat that everything worked perfectly in the office. Reactions to all the events generated by WebRTC and Kurento that I reached for were added to the code, and all this was written in great detail in the logs. I even put my logs in separate files so that they would not be confused with the main ones in the IIA.

Together with a keen doctor and client system administrator, we tried the system. They didn’t even test, namely, “tortured”. We created video conferences in all possible modes and from all available devices. Other doctors and part of the staff from the remote office were involved in this game.

The main thing was not to check the system (it did not work), but to collect as much data as possible. As a result, it turned out that:

  1. Approximately 80% of attempts to create a video conference are successful.
  2. Some connections using ICE candidates for IPv6 do not work.
  3. Out of 5 mobile operators, only 2 worked.

Everything turned out to be simple – you can’t go far on Google alone

Analysis of the collected information showed that the connection through the TURN server from Google is unstable. Either the load on them is large, or is it just a demo server for those who are just starting to learn the technology. But as a fact: very frequent failures. Need your own TURN / STUN server!

The second reason for the failures is the IPv6.local addresses. The kurento server does not accept ICE candidates with these addresses. It’s good that before sending all ICE candidates go through the code in my hands and I just filtered out IPv6.local.

The problem of mobile operators is solved, again, with its TURN / STUN server.
In three out of five mobile operators, NAT is symmetrical, and WebRTC cannot break through. More details can be found here: Is Symmetric NAT terrible?

It's a shame that my personal mobile phone works on a SIM card of an operator who did not bother with symmetrical protection. Therefore, my initial testing did not reveal this problem.

TURN / STUN server

The resiprocate-turn-server package was chosen as its server.

They didn’t choose for a long time – it is in the standard ubuntu repository – simple installation and automatic updates. But it’s not very convenient to work with accounts to connect: logins and passwords are taken only from the file, which is why you have to make an additional utility or script to export from the database of the main server.

At the moment, this file is generated by hand and accounts are distributed through a simple password pool. Authorization is implemented through the MIS main server, so security is not compromised. But the overall structure of the whole system looks ugly. The plans for remaking this moment.

Third trip to the client

I corrected the code, installed and configured my TURN / STUN server, developed a pool of passwords and distributed them to clients at the start of a video conference, and, after updating the production servers, I went to a doctor I already knew.

It works! Hooray! All conference launches are successful from all devices and in all modes: the patient from the personal account can call the doctor, the therapist during the reception can call the functional diagnostician for additional consultation, and the doctors themselves can arrange a multi-user video conference from different branches from all over the city.

Already taught by bitter experience, we engaged in meticulous testing with the artificial creation of emergency situations. From the topic of this article, I highlight the need to set a limit on the connection waiting time. WebRTC, along with Kurento, are waiting for the broadcast to start infinitely long and hope that the video bytes are about to go. I had to set a timer for 10 seconds, which gives an error to the managing server if the bytes never arrived.

After all the improvements

Finally, the system is functioning and working well. The first reviews from users in the field were sent. And immediately a huge number of wishes and suggestions for design, additional functions and other plans for further development appeared. The work began to boil with renewed vigor!

Now the full topology of the system looks like this:

RESULTS:

In conclusion, I want to say the following:

Firstly, WebRTC is an excellent technology with great prospects, but it is very difficult to test and debug it. Before starting development, it is imperative to deploy a network with all kinds of connections that the client may have. And debugging through the browser information window is not a very convenient tool.

Secondly, praise to Habru! While working on this project, I found a lot of information on this resource. All links in this article lead to it.

It was decided to leave the Telemedicine video-conferencing project itself to our organization’s support and development, we will not outsource it. In the future, there is still a lot of work:

  • There is a finalization of gluing recorded videos. I will again turn to Habr, I already found an article for a start: Combining video fragments from several cameras and synchronizing them in time.
  • It is necessary to redesign the user registration system on the management server, the main MIS server and the turn-server.
  • And the question is open with the throughput of the entire system as a whole. Stress tests have not yet been conducted. I’m getting ready, I’m reading it again Habr: How many participants can be in a WebRTC call?

EVERYTHING

I am sure that my experience will be useful not only for developers under WebRTC + Kurento, but also for those who are going to start implementing equally complex projects. Pay more attention to testing in conditions as close as possible to reality.

And take into account the risks that the support teams of your microservices can suddenly “disappear” – this is a very unexpected and unpleasant chore.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *