Cyara Solutions:
Chatbot Testing & Optimization
Don’t Let Your Chatbots Fail.
Take Them from Good to Great.
The use of chatbots is on the rise. Every industry, including ecommerce retailers, banks, and healthcare providers, are deploying chatbots for self-service to improve their customers’ experiences while keeping productivity up and costs down. Market size projections are off the charts with a projected $4.9 billion by 2032, with a CAGR of 19.3%, according to Precedence Research.
While many companies are enthusiastically optimistic about the role of chatbots as a customer self-service channel, consumers’ opinions on chatbots vary widely depending on their needs and past experiences. A recent commissioned study conducted by Forrester found that customers appreciate the convenience of 24/7 support; however, negative or frustrating chatbot experiences can push customers to seek out alternate – and potentially more costly – channels to interact with a brand.
Research by Gartner revealed that only 8% of customers chose to use a chatbot during their most recent engagement with customer service. And of those, only 25% felt that they would use a chatbot in future interactions.
Why are so many chatbots failing?
Some of the primary reasons a chatbot fails, or at least falls short of customer expectations, are that most chatbots do not understand customer intent, and many lack sufficient training and testing. And to exacerbate the situation, when chatbots cannot understand a customer, many fail to provide an easy escalation to a human.
For chatbots to be successful, they need to be more than a convenience. They need to deliver fast, accurate answers whenever possible, and they need to be able to move a request forward in a way that is most efficient and helpful to the customer. To accomplish this, enterprise chatbot development teams should focus their attention on the capabilities below.
Ensure Chatbots Understand Customer Intent
Customer intent is what the customer wants to get done, or the ultimate outcome they’re seeking when they engage with your company.
Examples of customer intent are returning a purchased item, receiving flight status, and replenishing funds for a debit card. Similar to Interactive Voice Response (IVRs), chatbots are challenged by understanding customer intent. For chatbots, there are many ways that humans express what they need in written format — that’s called customer input. Natural Language Understanding (NLU), a branch of AI, is often used by chatbots to interpret typed customer inputs.
For example, Andrea wants to return a t-shirt she purchased. Here is a small sampling of how she might type the request to a chatbot. Note that none of the requests uses the word “return,” yet the chatbot must understand that the customer intent is to return an item.
Example replies to chatbot: “t shirt too big and want $ back”, “hey, how do i send this back to u? can u pick up?”, “need a refund on this tee”
Understanding customer intent comes naturally to humans, but is extremely difficult to imbue in chatbots, particularly with the nuanced ways humans express themselves through customer input. Chatbots are only as good as the knowledge that powers them so development teams must collect large amounts of data that relate to customer intent.
Perform Chatbot Training
Chatbots need training that provides data on how to understand customer intent. Because of the complexity and diversity of ways in which humans convey their intent, it takes large amounts of data to represent the broad range of inputs that chatbots must be able to interpret. This training data is a set of examples expressing each intent that a chatbot uses to turn into a mathematical model for recognizing each intent.
Training data emphasizes target use cases, enabling chatbots to successfully recognize and handle each use case. Obtain training data from domain experts (conversational and customer service expertise), chat logs based on customers talking to chat agents, website requests, SMS, email, and any other documentation on how customers type requests.
In Andrea’s example above, to make sure chatbots are prepared to help her return the t-shirt that she purchased, development teams should collect the many ways that she might type her request, and include these in chatbot training data, like:
- t shirt too big and want $ back
- hey, how do i send this back to u? can u pick?
- Need refund on this t
- can i exchange for a small
- give me credit for t shirt
- hey amazon, take this back
Continuously Test and Optimize Chatbot Technology
Why? Because continuously testing chatbots assures quality at scale. Similar to other customer experience (CX) channels, chatbots require continuous testing and continuous improvement. In the previous example on Andrea’s request to return a purchase, the training data would be used to create testing data and test cases. There are increasingly new ways that humans express what they need, and automating the testing of your chatbot technology and using those insights to optimize your chatbot’s model is the only efficient way to continually learn and adapt to meet customers’ needs.
The DevOps methodology of shifting left directly relates to chatbot development. When teams adopt a shift-left mentality, it means they’re able to focus on problem prevention, rather than detection. As CX software is designed, testing is designed in parallel. With shift-left development, teams create test cases directly from designs, and can typically execute testing through the design phase.
To apply CX software best practices to chatbot testing, follow these critical steps:
- Create testing data
- Test target use cases
- Test non-target use cases
- Test chatbots in the context of the whole omnichannel customer journey
- Test chatbot escalations to a live chat agent
- Test chatbot and live chat agent performance under peak load conditions
- Monitor the chatbot in production and identify areas for improvement and optimization
Improve Accuracy & Containment for Chatbots and IVRs and Increase Customer Satisfaction – Powered by Conversational AI
Cyara’s CX Transformation Platform includes automated quality assurance designed specifically to assure, maintain and monitor your voicebot and conversational AI. Cyara helps you to deliver flawless customer experience through all channels and platforms by conducting automated NLP testing, conversational flow testing, security testing, performance testing, monitoring and optimization for your chatbot and models.
Chatbots are on the rise, but so are chatbot failures. Teach your chatbots how to understand customer intent, train your chatbot, and continuously test your chatbot technology to ensure that, if issues occur, you can catch and resolve them before your customers experience them.
Utilizing chatbots to cost-effectively and efficiently deliver a flawless customer experience is within reach. Make your chatbot an example of great customer experience. Learn how Cyara’s AI-Led CX Transformation Platform and Cyara Botium enable flawless CX journeys across voice and digital channels.
Assure Journeys. Transform Experiences.
Cyara revolutionizes the way businesses transform and optimize customer experiences. The Cyara AI-Led CX Transformation Platform replaces time-consuming, error-prone manual processes, empowering enterprises to drive CX transformation, monitor and assure customer journeys, and optimize CX across all self-service and agent-assisted channels, including voice, video, and conversational AI. Cyara helps many of today’s leading brands raise the bar on customer experience by ensuring that customer journeys perform exactly as designed, and that CX systems operate as intended, even under pressure. Cyara is the only choice for complete, end-to-end CX transformation!
Learn More
See how the Cyara AI-Led Customer Experience Transformation Platform can help you achieve your CX Assurance goals, and delight your customers.