This article was originally published on QBox’s blog, prior to Cyara’s acquisition of QBox. Learn more about Cyara + QBox.
According to current research, chatbots and conversational AI have been growing at a record rate during the pandemic. According to Research and Markets, “The global conversational AI market was valued at $5.78 billion in 2020, and is projected to reach $32.62 billion by 2030, registering a CAGR of 20.0% from 2021 to 2030.”
Learn how Cyara’s conversational AI optimization solution can help you assure quality and accelerate your chatbot development.
As organizations seek to automate processes, save costs on live agents and due to COVID-19, and overcome customer service staff shortages/absence, it’s essential to identify some quick wins that can get your chatbot on track to align with your 2022 goals.
Improve Chatbot Model Accuracy
Chances are if you’re using a chatbot as a first point of contact for your customers, they’re going to be redirected to live agents when the bot doesn’t answer a question correctly. Substantially reduce your fallback to live agent rate by getting your bot to return the correct intent first time.
If you are looking to get your model return the correct intents—it doesn’t need to be complex. The first step is getting a comprehensive understanding of how the NLP machine is interpreting your training data. Do you have one particular keyword in your training data that is distorting the result being returned? Perhaps you need to understand in more detail how each training utterance is influencing the result.
If you want to start 2022 by getting a look under the bonnet of your chatbot and improving chatbot correctness, it might help to know that it’s possible to understand exactly how each word in an utterance is influencing the intent returned. Our solutions can do all this and more—giving you clear indications of the results you can expect to be delivered not only for correctness but also confidence and clarity.
“More customers now get a correct response. By adding more intents, testing their performance, keeping an eye on the performance of existing intents, and slightly lowering our confidence threshold (which we could analyze through QBox), the fallback rate [has gone] from 20–25% to 10–15%.”
– Tim Lambrechts, SNCB.
Start Monitoring Chatbot Performance
SLAs, KPIs, ROI—businesses are full of useful acronyms that all boil down to one thing: how are we performing? So, is your business able to quantify the effectiveness of its chatbot? If not, perhaps it’s something to graduate to in 2022.
Setting minimum thresholds against what you aim to deliver—depending on how risk averse your organization is—is very helpful. Deciding whether you want to aim for your chatbot to answer 90% of customer questions correctly would help you benchmark your chatbot performance. We often find many organizations are not at all using confidence thresholds and our first advice is always to start. It will really help shape the continued improvement of your training data, especially when incorporating real customer utterances into your intents as you scale.
With our Confidence Threshold Analysis, you can determine either what percentage of questions you want answering correctly or how few you would like answering incorrectly. Our handy data visualizer will show you how well your data performs as per the most recent cross validation and what confidence threshold you should apply in order to meet your targets.
Great for you, great for your customers and…great when sharing your progress with top level management or cascading KPIs to those in your team.
Improve Experiences for Customers Using Your Chatbot
Are customers your guinea pigs for changes to your chatbot training data? Only too often we hear that for many, they have no other choice. You continually add intents to your model—perhaps reacting to a new product, service or circumstance that has created a requirement for new training data. You fix one intent, and you break another. But to find out where new issues are, you are setting your chatbot live again with your amends, only to work out later what is going wrong.
The truth is that every customer needs to have a positive experience in their interactions with you—and you need to minimize any scope for them to have a bad one. You need to have a chatbot performing well all the time, not just some of the time.
So how can you do this? Effectively training your chatbot is essential. Find out where you are causing regressions in your scaling without the need for customers to have those negative experiences. Testing the impact of new training data prior to go live removes the risk to both your reputation and your bottom line.
This is where our solutions can help. Not only can we show where your training data is falling short, causing conflicts, or not returning a clear result—once you have made changes to your data, it can show you exactly how those changes will impact your model. Spot those holes so your customers don’t have to.
Put Ongoing Care for Your Chatbot in the Hands of Subject Matter Experts
Who currently trains your chatbot? We’d bet that it’s not the person with the expertise in the product or services you are selling. Only too often, the responsibility for maintaining and improving chatbots is left entirely to technical staff—who are not close to the product or close to the customer. This results in a dialog that doesn’t always meet the need of the customer. It is also considerably more expensive to have data scientists and chatbot developers testing and maintaining chatbots. Not only does it take a lot of time and expertise to have these highly skilled staff maintaining your bot, they also on average have much higher salaries. Very expensive all round.
What you need is to bridge the gap between chatbot maintenance and the lower level, subject matter expert type members of staff. Something that is easy for anyone to use, visualizes data in a way that simplifies complex tasks into easy ones and clearly shows you when you’re making improvements.
This is exactly what our solutions are designed to do. It’s simple enough that anyone can use it. Don’t just take it from us. SNCB, Belgium’s national railway company, also said this: “QBox is an especially helpful tool for the SNCB team as most of the colleagues working with it don’t have a background working in IT. For our newest colleague, only some basic NLP model explanation was necessary before she could get to work with QBox, as the tool explained the rest.”
There you have it: Four resolutions to get your chatbot in tip-top condition, ready for all the uncertainty that 2022 might bring. If you’re in need of some more direct support, don’t hesitate to get in touch with us and see how we can help.
