how to increase your viewing time and understand that ML alone is not enough. Part 1
Immediately after the launch of the online cinema, the user is taken to the main showcase: there he sees popular films, new releases, the top of the week, and various selections. And there are also tabs “Movies”, “Series” – these are also showcases, but according to formats. Another variety is a television showcase. Right now we are actively working on it: we are moving from a vendor solution to our back one and have already launched an AB experiment.
Showcases consist of shelves. To make it more interesting for the user, we made several views:
Banner carousel is the very first shelf on the main screen. She has the most views;
VOD shelves (Video on Demand, that is, “video on demand”). The example in the screenshot above is “Especially for you.” That is, this is an ordinary shelf with video content, there may be films and TV series;
shelf with Originals. Here are films and TV series made by KION, and trailers are also played;
supershelves. Each of them consists of covers. They can lead either to another shelf or to a specific title – I’ll tell you what it is below, since you will encounter this word many times in this text.
Title is a unique designation for a piece of content, such as a movie or TV series. TV series have seasons, and seasons have episodes, and it's all one title. That is, a series is designated by one title, even if it has many seasons and episodes.
Let's get back to the topic. Currently the load on the service is 600 RPS. At the same time, we create a unique personal showcase for each user, and do not just “go” to a cache or database. That is, we prepare a personal search result for each request, apply more than 50 business rules – I’ll tell you more about them below. And all this in less than 300 milliseconds.
What is all this for? We want to grow such an indicator as TvTu — time per user in the system. And also increase conversion into conscious viewing and retention. In other words, we do everything to be watched more, longer and returned as often as possible.
ML in the KION online cinema
Now let's move on to machine learning. It gives +5% to viewing. Initially, we chose the evolutionary path of development. We run each feature through AB experiments, sometimes through ABC experiments.
What is an evolutionary path of development anyway? In the storefront, we select points of interaction between the user and ML. We define our goal. Next, we prepare the MVP, model, pen, conduct AB experiments, and look at the result. If the metrics look good, then we make a decision on how to roll out the finished product or feature into production.
For example, on the main storefront our goals are search, touchpoints, banner carousel, general VOD storefront. There is a shelf inside it. If you go to the movie card, there will also be a shelf here. This is the point of interaction between the user and ML.
How we collect a personalized offer
When a user launches KION or switches between storefronts, a request from the user comes to the back. KION has such a service – Blender: it works with user requests. Builds a storefront, searches for different sources, applies more than 50 business rules. For example, cut out viewed content or content that cannot be shown on this device. It happens that films cannot be shown on smartphones or tablets; they can only be shown on large devices at the request of the copyright holder. Then the request is returned.
We have another Media Gateway service in front of us, which proxies the request, receives our result and enriches it with information: links to pictures, ratings, purchase tags. Also, for example, it inserts a link to a banner with the required resolution for a specific device. That is, we don’t think about things like “what resolution should we display the banner in?” Media Gateway does everything for us. Blender assembles, composes the showcase, applies business rules and operates only on titles.
You need to understand that ML does not generate shelves, titles or make films – it works with the texture that already exists. We have more than a thousand shelves – ML takes ready-made shelves and personally ranks them both vertically and horizontally for each request.
Initially, the shelves were manual, but there is a problem: with such filling, films often “blink”. What does it mean? For example, the license for a film or cartoon has expired and disappeared from the shelf, then it may appear, and if, for example, there are less than three titles left in the shelf after filtering, it will be completely removed from the display case. We remove such short shelves, so the KION display case is always full.
Editorial shelves need to be monitored, so we came to auto shelves: the editors write the rules, we implement them. The service interprets this rule and fills the required shelves. There are already more than a thousand of them.
If a new title has been added to the system, our service understands that a new product has appeared. That is, the content, after transcoding and preparing all the banners, came to us through Kafka. We understand which shelves it needs to be placed on, we put it in, and it appears on the display.
There are simple shelves, for example “Russian Comedies” – categorizing content by genre. An example of a more complex auto shelf is “Popular Now.” For this, we take viewing statistics for seven days, analyze data from searches for seven days, and look at what is being searched for most. We combine all this into this shelf. And yes, I’ll add once again that it is personalized, that is, it is different for everyone. Well, we rank the content that we write there.
How to build a personal showcase
We have a template with “nailed” (fixed) shelves. They can be pinned differently for different platforms. When there are free slots, we make a request for a personal VOD shelf. Then we get the answer, filter it and arrange the already received shelves in the order that ML returned to us. “Nailed” shelves are also ML – for example, a banner shelf or “Popular now”. VOD shelves are placed in free slots.
Currently in production we have a model for VOD shelves, where the first three shelves are personal, and the rest are sorted relevantly by socio-demographic clusters. Personalization is becoming more and more relevant: users love variety and high ratings.
The symbiosis between man and ML has won
Not long ago we introduced ML into the banner shelf. We conducted several series of AB experiments – as a result, the symbiosis between man and ML won. So for the first five places for films on the main page, we put up content selected by our editors. On the “Movies” and “Serials” displays there are three places behind the editorial office. The rest is filled in by ML from those banners that exist throughout history.
When forming display cases and shelves, gradient boosting is used. Viewing statistics, user interaction, new features, trends, obsolescence and impression penalties are taken into account.
Features of novelty are three flags: “released today”, “this week”, “two weeks earlier”, that is, the model should throw newer content higher. And then there are trends.
Something about trends
Now let's talk about how we calculate trends. We take the conversion to conscious viewing (when a person came in and looked for more than N minutes) for a certain number of days. We calculate all this within the required period. Then we perform the same operation for the same period, only for the previous date, divide and get a trend. This is all fed into models for “learning obsolescence” or “penalty impressions”. When a new title comes out, it gets watched more, then less. If it's still new, it's thrown at the viewer until interest wanes.
We have viewing statistics: for all titles we count the difference between impressions and views – and feed this to the model’s input.
And the ML in the movie card takes into account popularity, meta tags, and uses the DSSM neuron. Our young scientists say that there should be two shelves – “Similar” and “Look also”, because these are different things.
That's all for today. I’ll be back soon with the second part of the post: in it I’ll tell you how we train neurons, what positive features we have and what difficulties arise, and also about how business rules work.
