This article was originally published on Botium’s blog on October 11, 2021, prior to Cyara’s acquisition of Botium. Learn more about Cyara + Botium
With the emergence of chatbots, many workplace roles have changed and in many cases, new ones have been born. As sad as it is, but a conversational AI tester is not one of them (yet). Of course, there can be several reasons behind this:
- Lack of resources (human, financial)
- Lack of expertise (knowledge in Test Automation)
- The general belief that Testing only comes into the picture after deployment (wrong)
- Management does not recognize the importance of QA
Therefore, Testing remains still in the background, as the “necessary evil”, which consumes resources and money.
It almost does not matter what technical or role background we had, Chatbot Testing will force us to leave our comfort zone, build up extra knowledge, and learn new expressions and methods.
We, at Botium, have decided to make things easier for companies that have already recognized the huge benefit of Testing Conversational AI, but have not yet had the opportunity to appoint a person for this task, or to gather the required expertise.
Although we believe that using Botium is already as easy as possible, (as there is no need to use Selenium or Appium), we have tried to facilitate the initial steps with maximum automation. Our new Go, Botium, Go feature gives you a huge advantage until you really have the necessary information to successfully set up your first test set.
A Quickstart, true to its name
The new Quickstart will not only give you simplicity and speed to get you started, but it will also help you to see what Test Types are possible for your chosen Chatbot Technology. Not all Chatbot engines forward the necessary information to run certain tests. For some, NLP testing is not possible as the Chatbot does not disclose the required information to run these tests, whereas in other cases E2E Testing is disabled because the Botium connection to the user interface (web widget, mobile app, or voice-controlled application) is done with dedicated Botium E2E Connectors.
Once connected to your Chatbot, Botium Box will automatically assign which Test Types are possible, tailored to the selected technology. The first steps remained largely the same. To kick things off you must select the Chatbot technology in the usual way and check the connectivity.
The biggest difference follows in the third step, where Botium Box automatically offers the Test Types that can be performed.
NLP and Regression Testing will be checked and run by default unless you intentionally skip it. These Test Types are the starting points for testing Conversational AI. When a Chatbot performs well in those disciplines, you do the next step and add Performance Testing, End-to-End Testing, Security Testing, and Monitoring.
The executable Test Types are shown in color, and those that are not supported by the selected Chatbot Technology are shown in gray. For certain Test Types that require some adjustment, Botium Box initially works with predefined values. These can of course be changed at any time with one click. As an example, we can see Performance Testing here, which can be changed to the desired values using the sliders.
Live Monitoring also works similarly. In case you enable this feature, you must enter the corresponding person’s or team’s email address(es) (who will be notified in case of complications) and the frequency of the live Monitoring to be executed.
Once you’re done with the settings, you get an overview of everything that Botium will execute for you. In case you are not happy with the results, you can easily go back to the previous page and change the settings accordingly.
And now, it’s time to make Botium work for you! Press the Go, Botium, Go button, take a sip of your coffee, and enjoy the instant results and gratification. The running tests are indicated with a spinning circle and the successfully executed tests are marked with a green checkmark.
By going on the question marks with your cursor, you get a cool tooltip to the executed test to have some deeper insights. In case you come across some unknown expressions, our Wiki is always there to give you a helping hand!
Congratulations, in 2 mins you’ve set up your first Test Suite and run it successfully!
Conclusion
Our latest feature, Go, Botium, Go simplifies the initial steps of Chatbot Test Automation and allows everyone to get some first insights about their Chatbot efficiency. In 4 easy steps, everyone will be capable of running tests and interpreting the results.
Please note that this can not replace continuous testing in an organized form, but it offers a huge advantage in terms of simplifying the start of your Chatbot Test Automation journey. The game-changer is no longer the ability, but the willingness!
