Quality recommendations in e-commerce – components of the main algorithms
In the first article of the series, we talked about the properties of quality recommendation algorithms that are necessary for the practical use of the algorithm. In the second article, we looked at the components of the “Related Products” algorithm. But the recommendation system for an online store consists of more than just the “Related Products” algorithm. There are other equally important algorithms.
In the final article of the series, we will look at several algorithms and analyze some of their components that are most important and different from those we examined earlier.
Recall that a component is a part of an algorithm that is responsible for certain functionality and can provide one or more properties. Components that solve similar problems for different algorithms may differ. This is explained by the fact that different algorithms help solve different problems of buyers. For example, the Cold Start component for the Related Products and Alternatives algorithms must find products that match each of those algorithms. For “Related Products” – products that complement the purchase well, and for “Alternatives” – options that can be considered instead of the current product.
As in the previous article, for each component we will tell you:
what problem is the component needed to solve?
how we solve the problem in our algorithm;
how to determine if there is a problem in the recommendation system;
what properties the component provides.
Algorithm “Alternative Products” and its components
The algorithm shows products that may be more suitable for customers. He helps those in doubt make a choice. When a buyer reviews a product, the algorithm suggests other options that may be better suited to them, which can lead to a purchase. This is one of the most commonly used algorithms on the product card page.
Using Buyer Behavior
The first thing you want to use as alternatives is products that have similar descriptions and attributes. The problem is that the most similar products are not necessarily the best alternatives. For example, if a buyer is looking for a TV and goes into a store to look at various options, he may find a TV that he is interested in, but does not fit the diagonal size. In this case, alternatives can be offered by other TVs, but with a different diagonal that is more suitable for it, which are often bought by people who were interested in the original product.
Let's give another example when the similarity in the description does not allow us to show the most suitable products. Let’s say that during Black Friday there was a promotion on a refrigerator, and they began to buy it more often. This product has not become more similar to the refrigerator that the user is interested in, but at the same time it has become a better alternative for him. A similar situation arises when a promotion passes and the product ceases to be in demand and is no longer the best alternative.
We have developed an algorithm that learns from the behavior of all customers and shows in recommendations the products that customers are most likely to add to their cart and buy after viewing the current product. If there is not enough data on additions to carts and orders, the model uses data on product views of all users. And only if such data is not enough, information on the similarity of descriptions and attributes is used.
To check the operation of the component, you need to find a block with recommendations for alternative products on the page of a popular product. Recommendations should suggest products that are similar in description to the one you are viewing, as well as those that are not similar to it, but may be a good alternative to it.
The component provides the following properties:
efficiency – the most suitable alternatives can be found;
specificity – behavior associated with the current product is taken into account;
relevance – recommendations change as consumer behavior changes.
Solving the cold start problem
If you build alternative recommendations solely based on data on customer behavior, then it will not be possible to formulate recommendations for all products when there is insufficient data for some of them. Most often this can be found with unpopular and new products. In the previous article we discussed this problem, however, we decided to note it here, since in different algorithms the components designed to solve the same problem can be arranged differently.
As a solution, we implemented a component that finds matching products using attribute similarity. This makes it possible to show recommendations for almost all products. In this case, products that have enough behavioral data will be shown first.
To check the functionality of the component, you can visit the cards of new or unpopular products and make sure that recommendations are displayed. In addition, you need to make sure that the products from the recommendations are alternatives to the current product.
The component provides the following properties:
The “Popular Products” algorithm and its components
When a buyer enters an online store, the first thing you need to do is to interest him. To do this, you can show him the best offers in the store. In addition, the buyer needs to understand as early as possible what types of goods are present in the store’s assortment. These problems are solved using the Popular Products algorithm. The algorithm is most useful on the site's home page, child and parent categories, but it can also be shown on other pages.
Increasing variety of products in recommendations
The most popular items in a store can be very similar. For example, in an electronics store, popular products may be smartphones, which will take up a large number of positions in the recommendations block. Since the number of products to display in the recommendations block is limited, there will be no room for products that could give a better idea of the store's assortment.
To solve this problem, Retail Rocket Group shows products from the largest number of different categories in the algorithm. The most popular products are selected for each category. For example, if there are more than 5 categories in a store, and there are 5 positions in the recommendations block, then one most popular product from the 5 most popular categories will always be shown.
It is easy to check the operation of the component. It is enough to visit the main page of the store and make sure that the block with popular products does not show many similar products.
The component provides the following properties:
efficiency – recommendations give a better idea of the store’s assortment;
coverage – more products from different categories are shown.
Using a category hierarchy
In the recommendations of popular products on the parent category page, you need to show products of child categories. For example, to show popular products for the electronics category, you need to know all of its child categories. The products may not belong to the electronics category itself and, if child categories are not taken into account, there will be no products in the recommendations.
Our system uses category hierarchy data provided by the online store to calculate the algorithm. Thus, recommendations for parent categories show products from child categories.
The component provides the following properties:
Increasing the impact of actionable data
Over time, interest in products may change and they may become less or more popular. This may be influenced by various external factors, such as seasonality, promotions, etc. For example, during the summer season, customers often ordered shorts, which became very popular in the store. And it may happen that with the arrival of the winter season, no other warmer product will gain the same popularity, which means there is a risk that shorts may be shown in recommendations in the wrong season.
In our algorithm, the newer the data, the more influence it has on recommendations. Moreover, the older the data, the slower their influence on recommendations decreases. For example, data that entered the system today and a month ago have different effects on recommendations. And data that was obtained a year and a year and a half ago has approximately the same effect.
Our system takes into account all available historical data. Old data has less influence, but they, like new ones, are also involved in calculating recommendations. To obtain the most relevant recommendations for the current moment, it would be possible to take only recent history, but then coverage for categories with products that are not popular at the moment will suffer.
To check, just look at the categories that contain products from different seasons and make sure that the popular recommendations correspond to the current season. Recommendations should also be shown in categories with unpopular products this season.
The component provides the following properties:
Using Different Event Types
Customers perform various actions on a store’s website: ordering, adding to cart, clicking on recommendations, and viewing products. The most important action is orders, but as a rule there are relatively few of them. On the other hand, a buyer may be interested in a product but not buy it, which can be an important signal for the algorithm. Thus, it is necessary to take into account different types of interactions and maintain a balance between their impact on the algorithm.
Our algorithm takes into account all interactions with a product, and each of them has a different impact on the popularity of the product. For example, a product that has been purchased 10 times will be more popular than a product that has been viewed 100 times. But if the second product is viewed much more often, then at some point it will become more popular than the first. Different types of interactions also lose their influence over time in different ways. We experimentally determined the optimal balance between the influence of different types of interactions on the popularity of products.
It is difficult to visually check the operation of a component, but sometimes you can use categories with cheap and expensive products to do this. Cheap products are more likely to be purchased, so they should be featured more often in popular ones. For example, in popular recommendations for the TV category, there should not be an overly expensive TV that is rarely bought.
The component provides the following properties:
coating – More data is used about different types of interactions, as a result, more categories will have a sufficient number of recommendations.
Filtering out data anomalies
Even one buyer can take too many actions on individual products and affect their popularity. Then products that are not actually popular may appear in recommendations. In addition, the popularity of products can be arbitrarily influenced by search engines and other robots.
In our algorithm, we limited the impact on popularity for each individual buyer over a short period of time. We measure popularity not by how many interactions a product has had, but by how many customers have interacted with a product over time. We filter traffic from bots completely.
If, despite the correct operation of all other components, you observe illogical products in the recommendations, then as a result of a detailed investigation of the problem, you will most likely discover an anomaly in the data.
The component provides the following properties:
Algorithm “Personal interests in categories” and its components
The algorithm determines categories that are of interest to the buyer in the long term. For these categories, it can show different products such as popular, new or other products. This is one of the personalization algorithms, and with its example we will demonstrate important features associated with personalization in general. The algorithm is most often used on the main page and on the parent category page.
The influence of categories on the formation of interest
Depending on the category with which the buyer interacts, interests can be formed in different ways. For example, after purchasing diapers or pet food, long-term interest will be formed in these categories. And after purchasing a smartphone or refrigerator, a person will not develop an interest in the category, since it is unlikely that he will soon need a second product of the same category.
For each category, our algorithm understands how a buyer's interest should change after interaction. Moreover, each type of interaction can have a different impact on buyer interest in the category. For some categories, the greatest interest is generated by viewing the category page, for others – by viewing the card, ordering, or adding a product from this category to the cart. However, it often happens that not all categories have enough data for the system to understand how to generate interest. Therefore, we collect data on all similar store categories and use them to determine how interest is generated.
You can check the operation of the component if you place an order for a product that is not in repeated demand, and make sure that products from the same category have disappeared from personal recommendations.
The component provides the following properties:
specificity – uses customer and category context;
efficiency – recommendations correspond to the buyer's task.
Generating interest in related categories
As previously written, interest in some categories may disappear after ordering. In this case, it is not obvious what can be shown in personal recommendations to customers. At the same time, the buyer may benefit from products that complement his order. For example, if you purchased a product from the phones category for which there is no repeat demand, then there is no point in showing other phones in personal recommendations, but you can show products that complement the order.
In our system, after an order, interests in categories can be formed, the products from which complement the previously completed order. For example, after purchasing a phone, you can show recommendations from categories such as cases, protective glasses and other accessories. In other words, the algorithm can suggest related products to previously purchased ones. Note that not every category makes sense to generate interest in related categories.
You can check the operation of the component if you place an order for a product that is not in repeated demand, and make sure that personal recommendations contain products that complement the previously purchased product.
The component provides the following properties:
efficiency – helps the buyer find products that complement the order;
coating – Interests in more categories are formed.
Calculation of personal recommendations in real time
If the buyer has taken an action and his interests change, then the recommendations should immediately change. For example, if a buyer was not previously interested in a product, but has now shown interest in it, then similar products should immediately appear in the recommendations. Otherwise, the recommendations will not help the buyer solve his current problem.
To solve the problem, our system receives behavioral data and changes customer interest in categories in real time. Adding complexity is that the system needs to store interaction data for each user over time. This is necessary because personalization should take into account not only actions within the current session, but also in past sessions, when a person leaves for a long time, returns and continues solving the previous task.
In addition to the interests themselves, the system calculates in advance the rules by which they change. It is impractical to perform such a calculation in real time, since all available data on the behavior of all store visitors is used, and the calculation turns out to be computationally very complex. Also, these rules change slowly over time, so they don't need to be updated right away.
To check the operation of the component, just look at some products in the store and make sure that similar products appear in the block with personal recommendations.
The component provides the following properties:
efficiency – recommendations correspond to the buyer’s current task;
relevance — recommendations are changed in a timely manner.
The “Accessories” algorithm and its components
The algorithm helps the buyer select accessories that complement the product.
For example, if a buyer chooses a smartphone, the algorithm will immediately offer him a suitable case, headphones, protective glass, etc. The main difference between accessories and related products is that related products can be used without each other, for example, pants and sneakers, while accessories are used only together with the main product.
Automatic detection of categories with accessories
The algorithm must know not only which products complement the current one, but also which of them are accessories. You can manually select store categories whose products are accessories. But this requires human resources, and constantly, since the categories on the site change, and, as a result, it is necessary to define new categories with accessories. In addition, with a large number of actions, a person can make mistakes.
Our system, without the participation of a marketer, automatically determines accessory categories and is able to show accessories that are suitable for the current product.
The component provides the following properties:
efficiency – the algorithm helps the buyer better solve the problem of finding accessories;
relevance – recommendations automatically take into account the emergence of new accessories;
coating – recommendations take into account more accessory categories and, as a result, recommendations can be shown for more products.
Upsell algorithm and its components
The algorithm recommends products that are as similar as possible, but with improved characteristics and more expensive. For example, if a buyer looks
wired headphones for 2,000 rubles and is going to order them, then the algorithm will show the buyer very similar, but wireless ones for 3,000 rubles, which will suit him no less than wired ones. A buyer may compare them, decide that wireless headphones will be more convenient, and buy a more expensive version.
At the same time, the recommendations of this algorithm may differ significantly from the recommendations in the previously discussed “Alternatives” algorithm.
Using product popularity
The algorithm can find many suitable products, from which it must select the best ones to display in recommendations. In this case, it is impossible, as in the alternatives, to use the preferences of other buyers who were interested in this product, since in this case products may be shown that do not correspond to the task of the Upsell algorithm.
To select the best products, we first show the more popular products that are most often preferred by store customers. For example, if a buyer is looking at clothes, and the style is the determining criterion for whether the clothing fits him, then the algorithm can find clothes of the same style, but from better materials. And this component will help to select from the already found products those that are most likely to be liked by the user.
To check, you need to go to the popular product card and make sure that the products that are shown by the algorithm must be popular in the current product category.
The component provides the following properties:
Conclusion
There are other recommendation algorithms in our system that contain many other components. In different algorithms, components can solve similar problems and work in a similar way, but each of them has its own specifics. In this article, we examined the most interesting components, the need for which may not be obvious at first glance.
In a series of articles, we identified the properties necessary for high-quality recommendations and showed that without these properties the problems of customers and businesses will not be solved. Typically, providing a single property is not very difficult, but implementing recommendations that have all the properties at once becomes challenging.
To successfully solve this problem, components are needed that can provide one or more properties, but not interfere with the provision of others. Moreover, each recommendation algorithm has its own characteristics, which leads to a unique set of components. All this makes developing a recommendation system very expensive.
We hope that if you are faced with the task of developing a recommendation system, then this cycle will help you understand what needs to be done to make the system suitable for practical use. And if you need to choose a recommendation system, then this series of articles will provide you with a way to evaluate its quality.
