In this article, we will touch on the topic of how you can improve what is already working well. We share our experience of changing recommender system algorithms to increase sales of one of the well-known retailers in the country. In business, recommender systems play an important role in increasing sales and conversions. Recommendations for products or services based on customer preferences and buying behavior can increase the average check and encourage additional purchases. However, it is important to periodically look for ways to develop the recommender system in order to be able to continue to improve the user experience.
In the article, we describe our real experience – a project to develop a recommender system for the Russian retail chain of supermarkets “VkusVill”.
Our previous article on predicting purchases based on previous experience
Questions to ask yourself
In order to understand how to move forward, one should turn to the origins – the key stages of the recommender system. The development of recommender algorithms can be divided into stages:
Definition of business goals, metrics, baselines;
Data collection and analysis;
The baseline can be either the current business recommendation system or simpler statistical heuristics – the recommendation of globally popular products, the recommendation of personally popular products, their combination, etc.
After the baselines on target metrics are beaten, and everything indicates that the recommender system works and is able to bring profit, the question arises – what to do next? Are there ways to develop a recommender algorithm or has the limit been reached?
These questions have fueled the development of more complex algorithms, using deep learning and artificial intelligence to create a more accurate and efficient recommender system.
Feedback accounting is the basis for the development of a recommender system
Why might this be necessary? There are situations when a well-functioning recommendation algorithm in some cases considers that a particular product is very relevant to the user, and therefore strongly recommends it. But the user is not specifically interested in this product and he ignores it for one reason or another. This is very useful information that is not available at the time of algorithm development, so it is necessary to start collecting such data at the testing stage. If, according to the test results, the algorithm performs well, information for a new iteration of its improvement becomes available to you.
Brainstorming and subsequent experiments
At one of the brainstorming sessions, we formulated a simple and quick-to-implement heuristic: we proposed pairing users and products, and giving such pairs an importance score, and then, depending on the importance, adjust the frequency of impressions.
For a better understanding, we propose to consider a synthetic version of our feedback on the example of the Mango product:
Pairs (customer, product) are grouped here and the last 5 impressions in these groups are displayed. We see that buyer AAAAAAA bought Mango 4 times out of 5 impressions, buyer BBBBBBB bought 2 times out of 5 impressions, and buyer CCCCCCC did not buy even once, no matter how many offers.
It was decided to introduce 3 types of ratings for pairs, reflecting the degree of importance of the product for the buyer:
Hot – if the buyer bought the product > 3 times out of the last 5 impressions;
Warm – if the buyer bought the product > 1 and ≤ 3 times out of the last 5 impressions;
Cold – if the buyer has not bought the product even once from the last 5 impressions.
We propose to consider the original synthetic vectors of recommended products for these buyers:
After applying the filter, taking into account the feedback, the recommendation vector will be as follows:
Nothing has changed for the AAAAAAA buyer, because he consistently buys it based on recommendations, so we conclude that the recommendation is relevant. For the BBBBBBB buyer, we have shifted the product in question further along the vector, which means, based on the technical implementation of our recommendation impressions, that in new iterations this product will be shown to the buyer later or less often. For the buyer CCCCCCC, we have removed the display of this product for a certain time. If you completely exclude non-purchasable products from recommendations, the buyer can very quickly find himself in a “recommendation bubble”. How long you need to hide such products, and then show them again, may depend on the requirements of the business.
By applying this algorithm based on a simple heuristic, we got enough gains on our business metrics to see the continued benefit of exploring more complex feedback accounting implementations.
Below are graphs comparing various business metrics of one focus group with feedback and two others without it.