Today, chatbots are one of the most common AI based enterprise applications. Most commonly used in support of customer care or for information retrieval, a chatbot is an application that is able to converse with a human via text or voice. A sophisticated chatbot can simulate a human-to-human conversation so well that it may be impossible for the human on the other end of the chat to distinguish whether they are communicating with a computer or a another human. Today, chatbots are becoming increasingly common in banks, public government, utilities and, more in general, in B2C enterprises.
Given their increasing adoption, the business expectations for what chatbots can or should be able to do have also increased. To avoid a potential backlash as a result of unrealistic expectations, I think it’s time to separate the hype from reality. In this post, I’d like to talk about some of the best practices to keep in mind for companies that are considering adopting chatbot applications.
The tech behind the chatbot
Chatbots have traditionally been divided into two groups. The first group is based on a set of rules that instruct the system on how to interact based on a user’s question and the context. The second group is based, in a broad sense, on Artificial Intelligence algorithms that helps the system get “smarter” learning from examples.
New developments in AI that have made, as an example, machine learning more available and affordable have contributed to the success of chatbots in recent years. Therefore, when we talk about chatbots today, we are mainly referring to systems that include AI algorithms that also have some natural language understanding capabilities. It is this latest generation that I’ll be focusing on today.
Best practices require a combined approach
The best possible performance of a chatbot does not depend only on the sophistication and quality of its ML algorithms and the related training but also on some best practices that can set the basis for a successful implementation. To keep things simple I will just focus on a few of these best practices so my intent is to not be exaustive but just provide some food for thoughts.
The first step would be the analysis of the scope of the possible and realistic questions that the chatbot could encounter from users. By excluding classes of question that would not address the needs of real customers, this initial analysis creates the basis for a more effective automatic understanding of all of questions that a user could ask. This somehow limited scope translates into greater accuracy in categorizing the incoming questions.
In a nutshell, the initial analysis allows us to create a taxonomy of types of questions that can realistically describe the majority of questions that the system will need to answer. It also limits the system’s exposure to other topics, which is necessary in order to prevent it from learning the wrong things. This is always a concern when it comes to automatic learning.
Another best practice for implementing effective chatbots is the constant focus on the quality of the knowledge repository from which answers to questions are retrieved. In this case, “quality” means that:
- The knowledge available is validated by a subject matter expert to make sure it contains reliable answers to questions that have been defined within the scope of the project, during the analysis.
- The knowledge is complete, meaning that it covers the full scope of questions defined as valid in the analysis phase with the same level of depth and reliability.
Even the best possible machine learning algorithm will not be able to provide satisfactory results if the two above conditions are not met.
Here are two examples of how this issue can impact the user experience.
Let’s look at a common, and seemingly basic, example: A customer asks about the opening hours of a bank’s branch office. Sounds easy enough, right? But let’s throw in a twist: In our example, the knowledge base of reference has the bank’s 2016 hours. But now we are in 2017 and the bank has changed the opening hours as result of a customers’ survey; to respond correctly, the system needs to know that. The user receives the wrong response and has a negative experience, but not because the AI system didn’t perform its task correctly.
Here is another scenario. This time, the knowledge base information is up to date and the system returns the correct opening hours. Now, the user is also interested in going to the branch office to ask for a consumer loan, a new product that the bank has just launched.
Even if the repository has not been updated with this new product, the system would still be able to correctly perform the automatic task of providing branch information. However, since the system has not been trained on the product information, it might offer a response like “product not found”.
How does this impact the customer? If she is aware that the bank does in fact offer this product, perhaps because of an ad she saw, she would assume that the chatbot doesn’t work. So, to get what she’s looking for, she will go back to a traditional channel and possibly conclude that the chatbot is unreliable and never use it chatbot again.
However, if instead she is not aware that the bank actually offers the product, she may seek out a competitor. The net result is a loss of customer confidence, a lost opportunity to gain a new customer, or both.
Conclusion
As we mentioned earlier, a chatbot application will not be successful if it depends 100% on AI. In limited, but relevant, scenarios above, the algorithm did exactly what it was supposed to do, yet it still failed in connecting the customer with the information they needed.
The human role is still a critical element in implementing a successful AI based chat application. The lack of human-driven analysis of the realistic scenarios and the construction and validation of a reliable and complete reference knowledge base would frustrate even the most advanced AI system. When it comes to enterprise AI applications, humans are still in the driver seat, for now and for the foreseeable future.
Author, Luca Scagliarini.
