As long as you have a green tick on the side, it means that the crawler successfully reached the end of the current path, without any failure. It suggests that the user will be also able to do the same in production.
Other cases with an exclamation mark are worth examining. In this case, the crawler could not reach the predefined depth in the conversation path. There could be more reasons behind this:
1. The conversation is stopped before reaching the maximum conversation steps
Before starting the crawler session there are several criteria you can add. One of them is the predefined number of conversation steps, that the crawler should reach. In case the specified criteria is not met, the crawler will not succeed.
2. Wait for prompt
You can define the maximum time a user should wait for a message from the chatbot. There are several conversation steps where the chatbot sends more than one request. In this case it is advisable to define a longer waiting time, but short enough to provide the feeling of a dynamic conversation. If the waiting time exceeds the predefined time limit, the conversation will be labelled with an exclamation mark.
Additional Crawler features
1. Exit criteria
You have the option to give a certain message as exit criteria. This means that when the crawler reaches this step, it will not go any further. This function is extremely useful, if your chatbot deals with a great amount of convos, but you’ve only done a tiny change in the conversation model and you don’t want to wait for the crawler to track through all conversation paths. If the crawler stops by meeting the exit criteria, it will be closed successfully.
2. Entry point
The conversation start message is a similar setting to the exit criteria, except that it does not specify when to stop the crawler session, but where to start. You may want to use this feature in situations where you have made changes at the end of a conversation model and it makes no sense to run the crawler from the beginning of the conversation. In case you establish an entry point, that the crawler can not find, the conversation will stop at the first step and it will fail.
Failing crawler session could also result in cases when the chatbot does not respond!
Botium Crawler will help you to identify missing paths and dead ends in the conversation flow and will help you to provide great user experience no matter what path the user takes.
The additional benefit of the crawler is that all detected conversation flows along all paths can be saved as Botium test cases and utterance lists and can be used as a base for a regression test set.
In the end Botium Crawler helps conversation designers to untie the inscrutable threads of human communications and to examine each conversation path as an individual part of the final user experience.