Unveiling the Value of Chatbot Testing

“What is the value of testing chatbots?” — A question that hits right in the heart of every passionate tester. Seeing it again in my inbox, it’s time to shed some light on this topic.

First – Chatbots are Software

According to the National Institute of Standards and Technology (NIST) Software bugs cost the U.S. economy $60 billion every year.

Second – The value of testing

Automation is the key enabler. Test automation leads to early identification of defects and unwanted behaviour. Decreasing the time to market as a result and the costs of quality while amplifying the end-user satisfaction level at the same time.

Third – The holistic test approach of Botium

The following image represents the BDLC (Bot Development Life Cycle). Milestones in every phase are shown underneath the gray line. Starting from the left with analyzing the user needs, deciding for your bot channels, followed by the NLP selection and so on. Above the gray line, you can see the holistic approach of Botium by adding value in every phase.



PLAN: Data sets for different domains help to identify actual user needs. Provider benchmark tests done with Botium Coach are the foundation to choose the right NLP engine for your bot.

DESIGN: Based on the conversation flow designed, Botium Box will automatically generate data sets for testing and training in just a few seconds. This a very good starting point to automate your chatbot.

DEVELOP: The Botium Crawler will crawl through the entire conversation tree of your chatbot and generate data sets based on the data collected. And if you add speech support to your bot, Botium has built in voice based testing.

TRAIN: The natural language processing and it’s NLP score are continuously challenged using Botium Coach.

TEST: Conversation flow testing, NLP score testing, real End-to-End testing, voice based testing, testing IVR systems, performance testing, security and GDPR testing just to name a few superpowers.

DEPLOY: Contantly keep an eye on your chatbot in production using Botium Monitoring.