voice and chat robot with artificial intelligence for you
Voice and chat robots have firmly entered our lives and no longer surprise anyone. We communicate with them every day, they make us appointments with a doctor, tell us the time of delivery of orders, and also advertise everything in the world.
But the main disadvantage of these robots is that they speak and perceive only predetermined phrases and nothing more. Very often this is not enough, so we try to quickly switch to a person.
Help fix the situation artificial intelligence. This area has advanced a lot recently and continues to develop at a high speed. There are already many so-called AI companions with whom you can communicate as with living people, almost not noticing the difference. Such robots learn in the process of communication, understand different phrases and try to answer differently even to the same question.
But what if you combine such an AI companion and a voice robot? As a result, you can get a voice assistant with which you can have a natural dialogue, like with a real person. And if you teach this assistant everything he needs to know, add realistic speech synthesis on top, indistinguishable from a real voice, the output is avatar from company Voximplant.
How an avatar works
Avatar is a service that uses NLP (native language processing) and artificial intelligence for voice or text communication with a live person. The avatar can be trained with any information, such as your coffee shop opening hours and delivery details, so that it can answer customer questions. Or you can integrate it with your CRM, and then the avatar will be able to help customers, for example, book tables in your establishment.
In addition, realistic synthesis and speech recognition can be added to the avatar, turning it into a very real interlocutor. Modern speech synthesis engines sound very believable, sometimes even indistinguishable from a real person, and AI and NLP make communication more natural. Add telephony or chat to this – we get a personal smart assistant for your hotline, contact center or website.
Create your own avatar
Let’s try to create a custom avatar that does all of the above. For this you need to have account in Voximplant. Go to the control panel, section Avatarsand press Create. Come up with a name for your avatar, select a language, time zone and create!
At this stage, you already have a ready-made and intelligent avatar that can analyze human speech and understand the speaker’s intentions. But this is not enough for us, we need to train him.
We train the avatar
To do this, open your avatar, go to the section intentions and press Create.
Intentions are what the avatar should extract from the speech of the interlocutor. For example, a customer may ask about opening hours. But since the question can be asked in different ways, the avatar needs training.
Open current intent section Training Phrases and write a few exemplary phrases of the client:
-
What are your working hours?
-
Until what time do you work?
-
When can you come?
Next, go to the section Answers and write response options for this intention.
Once the intent is saved, a yellow button will appear at the top of the screen Requires training.
We press learn and wait for some time. Artificial intelligence will analyze all the possible ways in which the client can ask this question, and in a real situation will know how to answer it.
In the same way, we add other intentions, for example, the possibility of delivering your products and the possibility of booking a table.
Next, we write a simple scenario communication with the client on the tab Dialogue script. The script is written in JavaScript. We need the avatar to greet the client, look for the intention to find out the opening hours of the establishment or delivery information in his speech, and be able to give an answer. I am using a script like this:
addState({
name: 'start',
onEnter:async(event)=> {
// когда клиент подключается, здороваемся с ним
return Response({utterance: 'Pineapple garden, how can I help you?', listen: true})
},
onUtterance:async(event)=>{
// ищем в речи клиента намерения
if (event.intent === 'openHours' || event.intent === 'delivery' || event.intent === 'reservation') {
// возвращаем клиенту ответ на его намерение и продолжаем слушать
return Response({utterance: event.response, listen: true});
} else {
// если намерение не ясно, просим перефразировать
return Response({utterance: 'Sorry, I didn\'t catch that. I can help you with open hours, deliveries, and reservations', listen: true});
}
}
});
// устанавливаем состояние начала диалога
setStartState('start');You can learn more about avatar scripts on the website. Voximplant.
It’s time to test what we’ve got. Click the button in the upper right corner debugto run the script. A chat window opens with an avatar:
We ask a question to check how intents are processed. The Avatar must detect intent in any natural speech and respond to it. We try:
Happened! The avatar detects the intentions of the client and answers the questions put to him. Now we need to teach the avatar not only to answer the client’s questions, but also to do something, for example, book a table.
Teaching an avatar to book tables
To do this, you need to train the avatar to recognize the “booking” intent using the method described above, and collect all the information for booking: date and number of people.
Since some of this information can be communicated already at the moment of intention, we begin to collect information immediately. That is, if the client says: “I can book a table for two”, then when this intention is detected, the number of people is already known.
Create a booking object in the script:
let reservationForm = {
slotTime: null,
slotPeopleCount: null,
uncertainUtterancesCount: 0
};We receive the necessary information from the client to fill out the form. If something is missing, ask specific questions. If the client could not answer the questions asked, increase the counter uncertainUtterancesCountand after three unsuccessful attempts, we stop asking to avoid looping.
When all the information is collected, we confirm with the client on what date and for how many people we are making a reservation. If everything is correct, we record the booking object and can send it to your CRM or backend via an API request.
I’ve added a few exit points to the script to keep it from looping, for example if the avatar didn’t understand the client three times, or if the avatar asked if the client could help and the client said “no” or said goodbye. The final dialogue script looks like this:
let reservationForm = {
slotTime: null,
slotPeopleCount: null,
uncertainUtterancesCountweirdUtterancesInRow: 0
};
addState({
name: 'start',
onEnter:async(event)=> {
// если попадаем в это состояние впервые, то здороваемся с клиентом. если нет, спрашиваем, чем ещё можем помочь
if (event.visitsCounter === 1) {
return Response({utterance: 'Pineapple garden, how can I help you?', listen: true})
} else {
return Response({utterance: 'Can I help you with somehting else?', listen: true})
}
},
onUtterance:async(event)=>{
// ищем в речи клиента намерения
if (event.intent === 'openHours' || event.intent === 'delivery') {
// отвечаем на заданный клиентом вопрос и переходим в состояние 'start'
return Response({utterance: event.response, nextState: 'start'});
} else if (event.intent === 'reservation') {
// клиент может начать сразу сообщать нужные данные
if (event.entities.systemTime) {
reservationForm.slotTime = event.entities.systemTime[0].value;
}
if (event.entities.systemNumber) {
reservationForm.slotPeopleCount = event.entities.systemNumber[0].value;
}
return Response({utterance: 'Sure!', nextState: 'reservation'});
} else if (event.intent === 'no') {
// если клиент скажет "нет, спасибо", то прощаемся с ним и завершаем диалог
return Response({utterance: 'Ok! Hope I\'ve helped. See you!', nextState: 'final'});
} else if (event.intent === 'yes') {
// если клиент скажет, что у него есть вопрос, спрашиваем и слушаем
return Response({utterance: 'Sure, so what\'s your question?', listen: true});
} else {
// если намерение клиента не ясно, переспрашиваем три раза, но не более
if (event.utteranceCounter < 3) {
return Response({utterance: 'Sorry, I didn\'t catch that. I can help you with open hours, deliveries, and reservations', listen: true});
} else {
return Response({utterance: 'I\'m so sorry, but I couldn\'t understand you. Bye!', nextState: 'final'});
}
}
}
});
addState({
name: 'reservation',
onEnter:async(event)=> {
if (reservationForm.uncertainUtterancesCount > 2 ) {
// если клиент не может дать ответов на вопросы или Аватар его не понимает, прекращаем задавать вопросы
reservationForm.uncertainUtterancesCountweirdUtterancesInRow = 0;
return Response({utterance: 'Sorry I couldn\'t understand you', nextState: 'start'});
} else if (reservationForm.slotTime && reservationForm.slotPeopleCount) {
// если вся информация получена, переходим к состоянию подтверждения
return Response({nextState: 'reservationConfirm'})
} else if (!reservationForm.slotTime && !reservationForm.slotPeopleCount) {
// если чего-то не хватает, спрашиваем
return Response({utterance: 'For how many people and which date would you like a reservation?', listen: true})
} else if (!reservationForm.slotPeopleCount) {
return Response({utterance: 'And for how many people do you need a table?', listen: true})
} else {
return Response({utterance: 'And for which date?', listen: true})
}
},
onUtterance:async(event)=>{
// проверяем, есть ли вся требуемая информация
if (event.entities.systemTime || event.entities.systemNumber) {
if (event.entities.systemTime) {
reservationForm.slotTime = event.entities.systemTime[0].value;
}
if (event.entities.systemNumber) {
reservationForm.uncertainUtterancesCount = 0;
reservationForm.slotPeopleCount = event.entities.systemNumber[0].value;
}
reservationForm.uncertainUtterancesCount = 0;
return Response({nextState: 'reservation'});
} else {
reservationForm.uncertainUtterancesCountweirdUtterancesInRow += 1;
}
if (event.intent === 'openHours' || event.intent === 'delivery') {
// если во время заполнения формы обнаружены другие намерения, даём ответ и продолжаем заполнять форму
return Response({utterance: event.response, nextState: 'reservation'});
} else {
// продолжаем заполнять форму
return Response({nextState: 'reservation'});
}
}
});
addState({
name: 'reservationConfirm',
onEnter:async(event)=> {
// приводим дату в понятную человеку
const months = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'];
const monthStr = months[parseInt(reservationForm.slotTime.substring(5, 7), 10) - 1];
const day = parseInt(reservationForm.slotTime.substring(8, 10), 10);
const hour = parseInt(reservationForm.slotTime.substring(11, 13), 10);
const minute = reservationForm.slotTime.substring(14, 16);
return Response({utterance: `So you want to book a table for ${reservationForm.slotPeopleCount} people at ${day} ${monthStr} ${hour}:${minute}`, listen: true});
},
onUtterance:async(event)=>{
if (event.intent === 'yes') {
return Response({utterance: 'Awesome! We will be waiting for you', nextState: 'start'});
} else if (event.intent === 'no') {
reservationForm.slotTime = null;
reservationForm.slotPeopleCount = null;
reservationForm.uncertainUtterancesCount = 0;
return Response({utterance: 'I see, sorry.', nextState: 'start'});
} else {
if (event.utteranceCounter < 3) {
return Response({utterance: 'I\'m sorry, so do you want to make a reservation?', listen: true});
} else {
return Response({utterance: 'Sorry, I can\'t help you. Hopefully I will be able to assist you next time. Bye', nextState: 'final'});
}
}
}
});
addState({
name: 'final',
onEnter:async(event)=> {
return Response({isFinal: true, needRedirectionToOperator: false, reservation:reservationForm})
}
});
// устанавливаем состояние начала диалога
setStartState('start');Now we have an avatar that can tell you about the company’s working hours and delivery options, as well as book a table for a specific date for the required number of people.
We connect telephony and chat
It remains only to give the avatar a voice and put him on the phone to answer calls. To do this, copy the code from the tab Integration our avatar, create an application on the Voximplant platform and paste the code into application script.
Setting up modules speech recognition And speech synthesischoosing a language and a voice you like from a variety of options, rent or connect phone number and set up call acceptance rule. Now the avatar is ready to communicate with customers!
Avatar can also be connected to text channel, for example, to the chat box on your site, where he will just as freely communicate with customers and do whatever is required of him. You can try to chat with the avatar in the chat in this demo on Node.js.
Outcome
We have made the simplest working scheme of an avatar that can communicate, answer questions and book tables, integrating with your system through API requests. But that’s just the simplest way to use it. An avatar can be taught a lot and make it an indispensable assistant for your hotline or contact center.
Voximplant’s speech synthesis and recognition modules allow you to select very realistic voices or integrate voices from third-party platforms. Thanks to them, your customers will not immediately understand that it is not a person who is talking to them. And constantly evolving artificial intelligence and NLP will make the avatar better every day!
Register on the platform and try to create your own avatar! Everyone who tests the avatar and leaves a review about it will receive a prize from Voximplant.
Do you have any questions?
Voximplant will hold a free webinar on the subject of avatars March 30 at 11:00 (MSK). On it you will learn:
-
how Voximplant Avatar helps create FAQ bots that unload contact centers;
-
are voice robots capable of making sales and not annoying the client in the process;
-
5 steps to develop a voice bot for developers with basic knowledge of JS;
-
how to create a voice bot that helps understand the delivery status.
There you can ask your questions and get answers in real time! Register for the webinar using this link: https://clck.ru/dnoNA
