Bots testing Bots

Automated Test Case Recorder – Introducing Botium Crawler

Welcome the new member of the Botium Stack – the Botium Crawler!

So you decided to add automated testing to your chatbot project ? Good decision 👍! But don’t underestimate the effort 😓 in composing a test suite with high test coverage from scratch. Botium Crawler to the rescue: It will do the hard work for you of setting up a boilerplate Botium test suite based on real chatbot dialogs.

Like a Website Crawler

A website crawler (a simple one) typically operates like this:

  1. It points to a website and analyzes the content
  2. It detects hyperlinks pointing to other sections of the website
  3. It navigates to all of the subpages, starting at point 1 again
  4. Until all subpages have been detected

Usage scenarios for a website crawler are:

  • Retrieve and prepare website content for search engines
  • Identify dead links (pointing to non-existing pages)

The Botium Crawler does exactly the same – not for a website, but for a chatbot:

  1. It opens a communication channel to one of the supported chatbot technologies (IBM Watson, Dialogflow, LUIS, Alexa, …) and starts a conversation
  2. It detects quick replies, hyperlinks, option lists presented by the chatbot
  3. It continues the conversation by sending all possible options back to the chatbot
  4. Until it reaches a point where there are no more options

Test Case Recorder

The Botium Crawler automatically detects pathes for navigating through the dialog structure. The following graphic shows the concept for an e-commerce chatbot and one of the possible pathes a user might navigate (red arrows).

Image for post

The Botium Crawler records all of the pathes and saves them as BotiumScript convos and utterances files. The recorded files can immediately be used for running Botium tests!

Usage Instructions

Botium Crawler is part of the Botium CLI now. With a single command, you send out the Botium Crawler to connect to your chatbot and detect the dialog structure:

Note: A botium.json file holding the Botium Core configuration is required

> botium-cli crawler-run --numberOfWelcomeMessages 2 --waitForPrompt 1000 --depth 5

With one more command, you can now run your first test suite based on the output from the Botium Crawler:

> botium-cli run --convos crawler-result/scripts

See the Botium Crawler Sample on Github!

Read on in the Botium Wiki!