NON-cool: PHP gRPC client in production

Hello! I want to show that getting gRPC in PHP is a normal combat solution that is quick to write, easy to deploy, and may be easier for you than sockets.

At first everything worked in REST and worked well, but growth began …

Skyeng constantly interacts with students: from time to time we need to call – to confirm the registration for a trial lesson, or, for example, to clarify if everything is fine if a person missed a lesson.

At the dawn of school, calls were handled manually, but the business quickly wanted to somehow automate and analyze the work of operators. Voximplant helped to do this. We still use their technologies – it’s convenient.

To prevent operators from listening to beeps, wasting time dialing unavailable numbers, and so on, the guys have a PDS (predictive dialing system) – an automatic dialing system. She takes two pools – operators and clients, and in the course of the call calculates the contact base, setting the speed of further dialing. The idea is for operators and customers to wait on the line as little as possible.

For a long time everything worked according to approximately the following scheme.

For example, we download a PDS list of 1000 numbers – and we know that we now have 50 operators. We start ringing the first 100.

• The number is not available – we process the script in JS, send the number and its status to the endpoint, write the fact of an unsuccessful call to the database.
• The answering machine works: everything is the same, but with the “answering machine” status.
• There is a call to the client – he picks up the phone and says “Hello”. At this time, the first suitable one is urgently looked for from the pool of operators: we know that if you keep a person “on the line” for more than two seconds, he will simply hang up.

With the growth, difficulties began. PDS calculates the contact base – and dials some more numbers. At the same time, it calculates the duration of current calls and updates statistics to balance the waiting time of operators and customers. The emphasis is always on the convenience of the client (those very few seconds), but the operator’s downtime should not be delayed. It’s okay if it’s 30-40 seconds. In practice, it happened that operators’ time utilization reached almost 30% – they sat waiting for a long time, this was critical.

We also began to encounter cases of double dialing. In the morning we collected a large call-list of thousands of numbers and uploaded it to the system. But in parallel, new potential students came through the site – again, empirically, the business established that they should be called in priority order in the next few hours after the application. PDS did not have the option to add numbers. Therefore, we stopped it, updated the sheet and started it again. But at the time of the stop, some of the numbers could go to the set – but their statuses have not yet arrived in the database. They were not marked as dialed and it happened that they talked to a person, and two minutes later the next operator dials him … We could not mark that the number was ringed on our side: since we worked with thousands of numbers at once, this caused big brakes on our base.

Then the guys from Voximplant gave us a prototype of their new PDS – a more advanced solution called PDS2. And we had to somehow connect to it.

Why choose gRPC? And why the Go client did not come

Oh, gRPC has a lot of cool features:

• Protobuf as a tool for describing the serialization of data types – we describe the protocol in a protofile, it’s fast.

Here’s a typical protobuf file – very similar to JSON, it’s pretty simple:

syntax = "proto3" ;

message SearchRequest {
string query = 1;
int32 page_number = 2;
int32 result_per_page = 3
}

• There is a gRPC plugin that allows you to generate all the necessary code from a protofile – service, types, PDS client, in our case. The only thing it doesn’t generate is the PHP backend.
• HTTP / 2 as a transport – you can stop the execution of a request on the server, and also reuse one socket for several parallel requests.
• Instead of typing “domain.com/service/collection/resource/request? parameter = value “, as in REST, there is only a service. Everything else is described through Protobuf in terms of our model and its events.

However, they chose not because of features: there was simply no choice – PDS2 communicated only via gRPC.

But with Go we tried it. The prototype client from the Voximplant developers has been around for a while, but it was hard to maintain. PHP is the main language in Skyeng, almost everything is written in it. And we realized that we need to drag a lot of code into the Go client, and then maintain it both in the client and in the PHP part. For example, we had problems with timezones – and there was a solution, but in PHP. And all this had to be carried over to that Go client.

We discussed this with the team lead for a long time, and in the end we decided that it would be easier to tighten everything in PHP. After months of operation, I understand that it was the right decision.

What if PHP was not delivered? Writing your own solution is (almost) simple

I followed the recommendations with gRPC.io for PHP. In principle, everything you need is described there.

There was only one funny nuance. I was looking for a solution for a couple of days on how to generate code with namespaces in our protofile. I generated everything, everything is fine, only they are not enough. As a result, I sat down to re-read all the documentation. It turned out to be called packages.

So, if you also ask yourself a question, everything is quite simple: we write in a file

package foo.bar;
message MyMessage {}

And this will generate a namespace like this.

FooBarMyMessage

Details on link…

How it works. When generating, we set the necessary parameters for connecting to Voximplant, start, and we get an endless loop that constantly listens to our stream. Our client is, in fact, an ordinary demon.

Here example from Voximplant. I can’t show our bundle: it has grown a lot due to the complex and specific logic for us.

What is the result

Our client, together with supervisorD, has been on sale since January, he is stable. In combination with a supervisor, this is almost a demon – if anything, the supervisor will raise and record the fall to himself.

We have solved the growth problems.

Thanks to the daemon, we ship numbers on demand, dynamically, in small portions – 50 at a time. And now, if we have some “hot” numbers from the face of the site, he already knows that these numbers have the highest priority – when a new request comes from Voximplamt, he sends them. We got flexibility.

And also, the waiting time for operators was reduced to about 20 seconds, but this is purely due to the best algorithms of the PDS2 itself, which was not written by us.