This article was originally published on Botium’s blog on April 15, 2021, prior to Cyara’s acquisition of Botium. Learn more about Cyara + Botium
Rule-based chatbots are a good starting point to gain experience in designing chatbots. These decision-tree-based bots are following a set of rules and use a planned, guided dialog.
Behind the apparent simplicity, there are plenty of operational-level difficulties for conversation designers in the maintenance. When a chatbot reaches a certain size, it becomes extremely difficult to handle and follow any changes in the conversation model. Logic jumps (to create different paths under conditions) and loops (to go back to a previous element of conversation) are giving you a wide range of flexibility in this closed environment, but they often make the conversation difficult to handle.
Once any change has been implemented, you have to make sure that it did not cause any confusion in other convos and that each conversation path stayed “healthy”.
Botium Crawler is here to help
Botium Crawler is the newest member of the Botium toolset. It was designed to imitate users going through all the possible combinations of the dialog structure at the same time. It automatically detects all conversation paths for navigating through the whole conversation model.
The following graphic shows the concept for an e-commerce chatbot and one of the possible paths a user might navigate (red arrows).
Botium Crawler simulates user clicks on all of the options in parallel, following all paths down until it reaches the end of the conversation or certain criteria.
In the flow option, you can see the visual representation of the conversation model of your chatbot.
This is something you can most probably see on your conversation design platform as well.
In the Crawler script view, you can see each convo path separately.
As long as you have a green tick on the side, it means that the crawler successfully reached the end of the current path, without any failure. It suggests that the user will be also able to do the same in production.
Other cases with an exclamation mark are worth examining. In this case, the crawler could not reach the predefined depth in the conversation path. There could be more reasons behind this:
1. The conversation is stopped before reaching the maximum conversation steps
Before starting the crawler session there are several criteria you can add. One of them is the predefined number of conversation steps, that the crawler should reach. In case the specified criteria is not met, the crawler will not succeed.
2. Wait for prompt
You can define the maximum time a user should wait for a message from the chatbot. There are several conversation steps where the chatbot sends more than one request. In this case, it is advisable to define a longer waiting time but short enough to provide the feeling of a dynamic conversation. If the waiting time exceeds the predefined time limit, the conversation will be labeled with an exclamation mark.
Additional Crawler features
1. Exit criteria
You have the option to give a certain message as exit criteria. This means that when the crawler reaches this step, it will not go any further. This function is extremely useful if your chatbot deals with a great number of convos, but you’ve only done a tiny change in the conversation model and you don’t want to wait for the crawler to track through all conversation paths. If the crawler stops by meeting the exit criteria, it will be closed successfully.
2. Entry point
The conversation start message is a similar setting to the exit criteria, except that it does not specify when to stop the crawler session, but where to start. You may want to use this feature in situations where you have made changes at the end of a conversation model and it makes no sense to run the crawler from the beginning of the conversation. In case you establish an entry point, that the crawler can not find, the conversation will stop at the first step and it will fail.
Failing crawler session could also result in cases when the chatbot does not respond!
Summary
Botium Crawler will help you to identify missing paths and dead ends in the conversation flow and will help you to provide a great user experience no matter what path the user takes.
The additional benefit of the crawler is that all detected conversation flows along all paths can be saved as Botium test cases and utterance lists and can be used as a base for a regression test set.
In the end, Botium Crawler helps conversation designers to untie the inscrutable threads of human communications and to examine each conversation path as an individual part of the final user experience.
