AI Ecom Assistant
I wrote an article about my thoughts about the AI Ecom Assistant.
For what? There are three reasons:
Nowadays, LLM and cool guys who are promoting these technologies to the masses, many are thinking about how to use this in their projects. I didn’t stand aside and thought a lot about what could be launched. The choice fell on AI Ecom Assistant. There is a large market, there is no well-known product that works well. I quickly realized that it was easy to do this if I had a lot of resources and brains, and I couldn’t handle the launch “by myself.”
I found a team/company that has the resources and plans to make such an assistant. I prepared this document, where I tried to describe my vision in detail. When writing this article, I tried to apply Amazon's approach to writing PRFAQ documents.
I think the document turned out to be good and quite complete; I didn’t manage to get into the team, but I wouldn’t want such a document to remain gathering dust on a shelf. Maybe it will be useful to someone or someone will be interested in it.
Before reading the article itself, I can’t help but share my thoughts about the PRFAQ approach. This is a useful approach to describing what you want to do. It is not clear why this is not used everywhere and is not more developed (at least in my world). What I was most pleasantly surprised by was the usefulness of the FAQ section. I showed the document to friends and they had questions. I sent the document to the team and they had questions. All you need to do is collect these questions and turn it into an FAQ section. This allows you not to rewrite the document itself every time, but to add it to the FAQ, which makes the document more complete and much more useful.
PR
“Search company” has released an AI shopping assistant! No more spending hours watching reviews on YouTube and reading articles on websites. The assistant will tell you about the important characteristics and help you choose the product that is right for you, compare prices and delivery conditions in marketplaces and online stores so that you can choose the most favorable conditions.
Intro
Imagine the best shopping experience you've ever had. There was likely a salesperson involved who showed just the right level of care for the situation. He stepped aside while you browsed the merchandise, but remained at the ready when you had a question. All the while following your prompts, it offered suggestions for products that you ultimately chose because the recommendations perfectly matched your tastes and criteria.
Can this type of personalized experience be replicated in a digital environment? It seems that technology has matured to the point of creating virtual assistants that can handle this task.
Problem
Let's say you decide to take up a new hobby – cooking steaks. Open any popular marketplace or online store and enter the query “steak grill.” Let's do this together, for example, look in ozon.ru: “The search for steak grill found 1,912 products.” The assortment is so large that it is not clear from the search results which of these grills solves my need best, which, in turn, comes down to the question: what set of characteristics should I choose (including price, delivery) so that the product solves my need, pleases me and creates feeling that this is what I need and I made the right choice?
Those. the problem, in my opinion, is not that there is a large assortment, and not that the product has many characteristics, but that you need to know and understand what characteristics are needed for the product to satisfy the user’s needs. Then, having selected the necessary characteristics, he will most likely receive a selection of several products and will be able to make a decision faster.
Continuing logical reasoning, the problem boils down to the fact that you need to teach the user to understand the characteristics of the product and understand what problems they solve. Reducing the time for studying is the main task that the assistant must solve, which will ultimately reduce the time from awareness of the need to purchase.
Competitors
Before we talk about the solution, let's look at the competition. Yes, of course, direct competitors are marketplaces: Ozon, Wildberries and even Yandex Market. Filters, ratings, reviews, guides – all this helps solve the problem of choice.
I especially want to talk about indirect competitors: YouTube, Instagram, review sites. There, bloggers do the same work: they studied the characteristics, told which ones were important, talked about the features and suggested the best products, in their opinion.
Solution
So, what might a solution to the problem we outlined above look like?
It seems to me that it is correct to act as a “good” sales assistant in a store acts. If we summarize the logic of his actions, he iteratively, from general to specific, clarifies the buyer’s needs and at each iteration step shows several different products, the best in their category, and tells how they differ from each other. This allows the buyer to better understand the important characteristics and understand which ones are important to him.
Firstly, I would like to draw attention to the idea of “going from the general to the specific.” This approach will allow you to gradually immerse yourself in a category to study it, rather than immediately dumping all the data that exists on the user. This will make it easier to absorb information.
Secondly, on the proposal “show several different options at each step.” It’s easier to understand when you can compare something with each other, compare it with an example. The greater the differences between products, the better this example explains the essence.
And thirdly, “shows several different products that are the best in their category.” I think it's very important to start showing the product as early as possible. It is necessary to combine a rational approach to choosing a product with an emotional and impulsive type of purchase. To help you decide on the right product as early as possible, not all of us want to dig deep into the details.
The ideal solution that you want to come to is that you no longer need to go to watch reviews on YouTube, you don’t need to go to websites to read reviews, you don’t need to go to marketplaces and online stores to study the assortment and reviews. All this can be done through the AI assistant. He will tell you about important characteristics and help you decide on those that you need, analyze reviews and tell you about the features, help you choose a product and find the place with the most favorable conditions for purchase.
How it works
Before moving on to describing how the solution might work, I want to highlight a few important points. First, it seems that a purely chat interface is not the best way to browse and select products. Second: I would probably consider the AI-assistant as an AI-Copilot for the trading vertical. And third: it’s interesting to spin the voice script.
Let's design a possible abbreviated CJM, try to choose equipment for cooking steaks at home. Judging by Wordstat, very few users directly write such a request for “equipment + for steaks” (66); they mostly write “grill + for steaks” (4314), “electric grill + for steaks” (651).
Action | Result |
Enter “grill + for steaks” into the search. | In the search, an assistant card appears with a call for help in choosing a grill. Based on the logic described above, we show the categories on the card: electric, gas, coal and the “best” products from each category. |
Click on the card | We move to a separate vertical, where we show already familiar categories with the “best” products. We write down the difference between the categories, what are the pros and cons, and for which scenarios it is better suited. Please choose the one that suits you best. The “best” product is clickable, you can go straight to look at the description and look at the places where the best conditions are. |
Select the “electric” category. | The next important characteristic for cooking steak is “power”. We divide categories into different ones according to power and explain how power affects the task with which the user came. Categories: up to 1800, from 1800 to 2200, from 2200 And still, we immediately show the “best” product in each category that you can buy without worrying about it. |
Select a category from 2200 | And so on … |
The solution is superficial, but I wanted to demonstrate the essence and approach. If you imagine, you can understand that it combines the usual format, when we show ratings on products, there are reviews, but in addition to this, it also carries the format of selections from bloggers or reviews, when we show a selection of categories and “best” products, selected by us for the user.
Success Criteria
We have already discussed that reducing the time spent studying characteristics and understanding what tasks they are needed to perform is the main task that the assistant must solve. Time to learn is a poor good metric. Still, the ultimate goal is to quickly bring the buyer to a purchase by reducing the time spent on research.
The main metric that reflects how well a service helps sell a product is the generally accepted GMV metric. I will measure success or failure by this metric. If a product grows this metric across all searches, it means it's providing value to users, search, and stores.
What other metrics would I look at and what would I collect in order to understand what is happening in the product:
DAU / WAU / MAU – the number of users who use the product.
Assist Share – what share of Ecom requests the assistant covers. We consider the metric as the ratio of the number of assistant impressions to the number of Ecom requests.
Assist Buy Funnel – a complete funnel from display in search to purchase. It is most likely difficult to log the fact of a sale, but since we can calculate GMV, it means that with some probability we can definitely do so. If the fact of sale is a very noisy metric, then we will look before going to the store, but it would be good to record the fact of sale. I'll break the funnel down into steps:
Activate Conversion Rate – the percentage of activations, calculated as the ratio of the number of transitions to the AI assistant to the number of impressions in search.
Good Conversion Rate – the percentage of clicks to a product, calculated as the ratio of the number of clicks on the product card (display of the product page) to the number of clicks to the AI assistant.
Offer Conversion Rate – the percentage of conversions to the store, calculated as the ratio of the number of transitions to the store page to the number of impressions of the product page.
Deal Conversion Rate – percentage of purchase, calculated as the ratio of the number of purchases to the number of transitions to the store page.
Steps AvCount – the average number of steps/dialogues that the user took to go to the product, which can be reformulated as: find an interesting or suitable product.
Buy, Price – I really want to log the fact of purchase and the purchase amount in order to calculate “Number of purchases”, “Number of repeat purchases”, “Average number of purchases per user”, “Average bill”. All these metrics, as you understand, directly affect GMV.
Retention Rate is the percentage of users who used the AI assistant and returned to the search script and used the AI assistant again.
I chose these metrics, I consider them the most important. The remaining metrics, which also need to be monitored, are either used to calculate these, such as “number of transitions to assistant”, “number of product impressions”, etc., or will already be submetrics within these, such as “number -in dialogues”, “average bill per user”, “CTR position of the store offer card”, etc., and will influence these main ones.
Advertising
If this product helps increase search usage in ecom scenarios by increasing user value, this could have a positive impact on advertising revenue. Now it is not yet clear how to make friends with advertising. Maybe this will be another commercial ad nearby in the selection. Maybe already on the product page? Maybe we’ll come up with something else, and most likely more than once.
Advantages and Disadvantages
A search company has a very important advantage in solving this problem, which is not available to other players: we are actually a search engine and know about all online stores and their range/prices. This allows us to make a more complete and competitive comparison of offers to choose the best option.
Among the shortcomings, I would highlight the fact that not all conditions are available to us. For example, we do not know the delivery conditions, and this is an important characteristic in making a purchasing decision. Another disadvantage is that you can’t buy it right here and now, without going anywhere and understanding the new interface.
Strategy
More and more people are going directly to marketplaces and stores through apps to solve their product scenarios, bypassing search. As a result, stores receive fewer orders from search and transfer their advertising budgets to marketplaces and other channels, and the company earns less.
One of the important properties of a product is what users associate it with and whether they remember it when they have a need. People are already accustomed to Ozon and Wildberries and remember that they need to buy there. Habit is the hardest barrier against which many products “break.” An assistant who lives in search is one thing, but an assistant as a separate entry point to which you can come for any scenario is another. That's why I like T-Bank's approach: “Universe of AI assistants” This is not a call to action, it is more about identifying risks. I suggest thinking about this and keeping in mind that you need to develop a separate entry point for the assistant(s).
A full-fledged AI assistant is not available to everyone; it is expensive. This is a good opportunity to offer this solution for B2B; you can provide assistance to third-party stores.
FAQ
Interior
Maybe it’s better to first clarify the user’s request, and only then show the final selection?
This approach has been around for many years, it’s called Quiz, and you can find examples of projects in the additional materials. I don’t see that this is a very mass approach, from this I conclude that this is not the best solution.
On the other hand, if you look closely, the described solution is also Quiz, only modified. Instead of blindly and boringly choosing characteristics, I propose to iteratively, step by step, tell what this or that characteristic affects, and demonstrate the difference using examples of products. I think this approach is more visual and less boring.
What are the best products at each assistant stage?
Right now, when a user enters the search term “steak grill” or uses filters to select the desired characteristics, products will appear to him. What products will be in first place? Apparently, these are some frequently sold products in this category, with a good rating and an average price/quality ratio. Now such output is “prepared” by neural networks; we do not know exactly what parameters influenced this output. My point is that this problem is already being solved, and the solution may be the first product in the search results, based on the given characteristics.
Which proxy metric for GMV can be from the ones you specified?
What to do with the fact that GMV is difficult to paint on such goods? These are expensive long-term purchases. And they are often postponed.
I completely agree that GMV is difficult to paint, especially as part of experiments. What can be a proxy metric?
On the one hand, such a metric could be the number of transitions to stores. The more we generate transitions to “stores”, the greater the chance of purchase, the greater the GMV. What if we now select and explain so well that now there is no need to go and look at many different products, the user goes through less often, but buys more often?
On the other hand, what worries Search the most? The biggest concern is that search traffic is going to marketplaces. They stop using search. Then the growth of search traffic on Ecom slices is an excellent metric that reflects success. What if we began to solve the scenario poorly and the user began to return to the search more often?
Choosing a solution based on metrics is often a trade-off: you improve some things, worsen others. We need to find a balance. As proxy metrics, I would look at both the number of search queries and the number of clicks to stores. I think they complement each other well.
On which queries should the functionality be shown?
On requests whose classification is suitable for Ecom and for which we have an assistant. The Ecom classification of queries should include the name of the product category (grill, refrigerator, …), additional marker words (buy, choose, as a gift, …). The more obvious the intention for purchase and choice, the more interesting this request is for us.
Where do we get data for such dialogues?
When I experimented myself, I started by making myself GPTs (what is GPTsand here's mine ShopMate AI GPTs), which operates in a similar way to the logic I described above. Through it you can type the first examples of dialogues and understand the applicability of this approach.
For a solution that will already be tested on users, I would suggest the following plan:
I would start creating my data. You can hire real sales consultants and ask them to answer user questions in real time and offer products from the market. One hundred dialogues for each product category would be enough.
I would try to find ready-made dialogues: ready-made datasets on the Internet, logs of correspondence in chats on Ecom sites, recordings of conversations between sales consultants. How many such dialogues would you like to collect? Of course, the more the better, but at this stage the quality of such dialogues is more important.
All collected data must be anonymized and structured. Make a quality mark and leave the best ones. The markings would be done by assessors (by previously hired sales consultants). The criteria will be naturalness, relevance, connectedness, usefulness. Each criterion will need to be assessed on a five-point scale.
I would additionally train a larger model using the collected data. It would generate even more synthetic data: 10K-100K for each type of product. It is possible using different persons. I read about it good articlethis approach seems reasonable.
Synthetically collected data needs to be whitewashed and labeled. Only manual marking will not work here. For such cases, a set of high-quality real dialogues is selected (we have them) and evaluated using the metrics BLEU, ROUGE, Perplexity. I would take the best dialogues based on metrics data into a training dataset.
I would have retrained the model using all the available data. I would generate synthetic dialogues and measure their quality. I would take some of the dialogues for marking with the help of assessors in order to check the quality of the dialogues by people.
This option would roll out to users in alpha/betta and begin to collect real dialogues on a larger scale. I would mark up the dialogues further and monitor the quality. I would add more factors, such as user rating of the dialogue, clicks on product cards.
I would retrain the model using new selected real dialogues, and so on.
Custom
Why will this be more convenient than filters and sorting?
Because filters and sorting begin to work when the user knows what this or that characteristic is responsible for and which of them are important. If he is an expert, then he does not need an AI assistant and can use filters and sorting – this way, most likely, you will choose the product faster. But if you are not an expert, then filters and sorting are an ineffective tool. You need to go and study information about the desired type of product, become an expert, and this takes a very long time. The assistant will help you figure it out faster and you don’t have to go anywhere for additional knowledge.
Why is this better than going straight to a marketplace or online store?
Our main advantage is that we, as a search engine, know about products on all marketplaces and online stores, so we have a larger assortment than any other site. In addition to products, we know what prices these products are selling for on different sites, which allows us to find the best deals.
How is this better than blogger reviews on YouTube or websites?
Firstly, a good review on YouTube still needs to be found, and before this happens, you will have to spend a lot of time. Secondly, bloggers can add products from manufacturers who paid them for it to reviews in order to advertise the product.
We have already tried to analyze YouTube reviews, reviews with ratings on websites and are showing a summary based on various sources. Such information is more “honest”, complete, and you don’t have to waste time searching for this information somewhere else.
