• Skip to primary navigation
  • Skip to main content
  • Skip to footer
Cyara logo

Cyara

Cyara Customer Experience Assurance Platform

  • LOGIN
  • CONTACT US
  • WATCH A DEMO
  • PRODUCTS & SERVICES
    • Automated CX Assurance Platform
      • Call Explorer
      • Call Routing & Agent Desktop Testing
      • Global In-Country Dialing
      • Integrations
      • Omnichannel Testing
      • Voice Quality Testing
    • Products
      • Botium
      • Cloud Migration Assurance
      • Cruncher
      • Pulse
      • ResolveAX
      • Toll-Free Fast Check
      • Velocity
    • Services
      • Cyara Academy
      • Consulting
      • Customer Success
      • Expert Services
      • Support
  • SOLUTIONS
    • IVR Testing
      • IVR Discovery
      • IVR Testing
    • Omnichannel Testing
      • Chatbot Testing
      • Cloud Contact Center
      • Contact Center Testing
      • Email & SMS Testing
      • Omnichannel Testing
      • Web Interaction Testing
    • Software Testing & Monitoring
      • Continuous Testing Solutions
      • Customer Experience Monitoring
      • DevOps for Customer Experience
      • Functional Testing
      • Incident Management
      • Load/Performance Testing
      • Regression Testing
    • Voice Quality Testing
      • Agent Desktop Testing
      • Outbound Call Testing
      • Voice Biometrics Testing
      • Voice Quality Testing
  • RESOURCES
    • Blog
    • Events
    • Customer Success Showcase
    • Resources
    • Webinars
  • ABOUT
    • News
    • Partners
    • Customer Awards
    • Company
      • About Cyara
      • Careers
      • Employee Profiles
      • Industry Awards
      • Leadership
      • Legal
  • English

Chatbot Testing Solutions

Cyara Solutions:
Chatbot Testing

Don’t Let Your Chatbots Fail.
Take Them from Good to Great.

The use of chatbots is on the rise. Every industry, including ecommerce retailers, banks, and healthcare providers, are deploying chatbots for self-service to improve their customers’ experiences while keeping productivity up and costs down. Market size projections are off the charts with a projected $1.34 billion by 2024, and a 31% CAGR. 1

While companies are excited to usher in chatbots as the new customer service channel, consumers are far less enthusiastic. According to Forrester, 54% of consumers expect a chatbot to negatively affect their lives. 2

Why are so many chatbots failing? Some of the primary reasons are that most chatbots do not understand customer intent, and many lack sufficient training and testing. And to exacerbate the situation, when chatbots cannot understand a customer, many fail to provide an easy escalation to a human.

For chatbots to be successful, they need to be helpful to customers. To achieve that status, enterprise chatbot development teams should focus their attention on the capabilities below.

Ensure Chatbots Understand Customer Intent

Customer intent is what the customer wants to get done, or the ultimate outcome they’re seeking when they engage with your company.

Examples of customer intent are returning a purchased item, receiving flight status, and replenishing funds for a debit card. Similar to Interactive Voice Response (IVRs), chatbots are challenged by understanding customer intent. For chatbots, there are many ways that humans express what they need in written format — that’s called customer input. Natural Language Understanding (NLU), a branch of AI, is often used by chatbots to interpret typed customer inputs.

For example, Andrea wants to return a t-shirt she purchased. Here is a small sampling of how she might type the request to a chatbot. Note that none of the requests uses the word “return,” yet the chatbot must understand that the customer intent is to return an item.

  • t shirt too big and want $ back
  • hey, how do i send this back to u? can u pick?
  • Need a refund on this tee

Understanding customer intent comes naturally to humans, but is extremely difficult to imbue in chatbots, particularly with the nuanced ways humans express themselves through customer input. Chatbots are only as good as the knowledge that powers them so development teams must collect large amounts of data that relate to customer intent.

Perform Chatbot Training

Chatbots need training that provides data on how to understand customer intent. Because of the complexity and diversity of ways in which humans convey their intent, it takes large amounts of data to represent the broad range of inputs that chatbots must be able to interpret. This training data is a set of examples expressing each intent that a chatbot uses to turn into a mathematical model for recognizing each intent.

Training data emphasizes target use cases, enabling chatbots to successfully recognize and handle each use case. Obtain training data from domain experts (conversational and customer service expertise), chat logs based on customers talking to chat agents, website requests, SMS, email, and any other documentation on how customers type requests.

In Andrea’s example above, to make sure chatbots are prepared to help her return the t-shirt that she purchased, development teams should collect the many ways that she might type her request, and include these in chatbot training data, like:

  • t shirt too big and want $ back
  • hey, how do i send this back to u? can u pick?
  • Need refund on this t
  • can i exchange for a small
  • give me credit for t shirt
  • hey amazon, take this back

Continuously Test Chatbot Technology

Why? Because continuously testing chatbots assures quality at scale. Similar to other customer experience (CX) channels, chatbots require continuous testing and continuous improvement. In the previous example on Andrea’s request to return a purchase, the training data would be used to create testing data and test cases. There are increasingly new ways that humans express what they need, and automating the testing of your chatbot technology is the only efficient way to continually learn and adapt to meet customers’ needs.

The DevOps methodology of shifting left directly relates to chatbot development. When teams adopt a shift-left mentality, it means they’re able to focus on problem prevention, rather than detection. As CX software is designed, testing is designed in parallel. With shift-left development, teams create test cases directly from designs, and can typically execute testing through the design phase.

To apply CX software best practices to chatbot testing, follow these critical steps:

  • Create testing data
  • Test target use cases
  • Test non-target use cases
  • Test chatbots in the context of the whole omnichannel customer journey
  • Test chatbot escalations to a live chat agent
  • Test chatbot and live chat agent performance under peak load conditions
  • Monitor the chatbot in production

Chatbots are on the rise, but so are chatbot failures. Teach your chatbots how to understand customer intent, train your chatbot, and continuously test your chatbot technology to ensure that, if issues occur, you can catch and resolve them before your customers experience them.

Utilizing chatbots to cost-effectively and efficiently deliver a flawless customer experience is within reach. Make your chatbot an example of great customer experience. Learn how Cyara’s Automated CX Assurance Platform and the Cyara Botium product enable flawless CX journeys across voice and digital channels.

Customer smile in magnifying glass

Learn More

See how the Cyara Automated Customer Experience Platform can help you achieve your CX Assurance goals, and delight your customers.

Watch a Demo

1 New Chatbot Market 2018-2024, Market Study Report, December 2018.

2 Ian Jacobs, Julie A. Ask, Andrew Hogan, Forrester Infographic: Customer Service Chatbots Fail Consumers Today, Forrester, January 30, 2019.

Footer

Cyara logo
Contact Us Kontakt Nous Contacter
  • LinkedIn
  • Twitter
  • YouTube

Copyright © 2006–2023 Cyara® Inc. The Cyara logo, names and marks associated with Cyara’s products and services are trademarks of Cyara. All rights reserved. Privacy Statement

  • English