What makes a great chatbot?
What we know for sure from market research is that functional features are undoubtedly the most important for users. Functional features are responsible for a product or service to reach its primary objective. We list some features here that directly serve the chatbot to fulfil the task at hand.
Read MoreChatbot Testing: the ultimate business benefit
In this article, we highlight the main business reasons that make automated chatbot testing with Botium an essential and unmissable part of every chatbot project.
Read MoreContinuous Speech Recognition Testing
Botium Box enables companies to implement a holistic test strategy for voice assistants on all levels of the component stack. This article focuses on the speech recognition part and shows how to build a regression test suite based on audio files, transcription files and word error rate verification.
Read MoreTop 3 Chatbot Security Vulnerabilities in 2022
Since chatbots offer a wide range of applications, in certain cases they become responsible for collecting and protecting personal information as well. Consequently, they are a great attraction for hackers and malicious attacks too. The responsibility of ensuring chatbot security has become more pronounced after the introduction of GDPR in Europe. As statistics show that this technology will be a determining factor in our lives, security testing must also become part of our daily tasks, so that these chatbots can be used with confidence.
Read MoreFirst steps in Chatbot Performace Testing with Botium Box
One major pitfall of building chatbots is underestimating the importance of performance. The UI of a chatbot is usually very simple, so it’s easy to forget the complexity behind these virtual assistants. A slow chatbot might be accepted for home projects, but a company can not neglect it. Bad performance is a serious UX killer.
Read MoreHow to set up the fundamentals of your Chatbot Testing in 2 minutes
Although we believe that using Botium is already as easy as possible, we have tried to facilitate the initial steps with maximum automation. Our new Go, Botium, Go feature gives you a huge advantage until you really have the necessary information to successfully set up your first test set.
Read MoreThe biggest pain points in using chatbots
Chatbots are — lead generators, help desks, customer services, reinforcements for brand recognition and so on… but what if customers intentionally avoid talking to your bot? What are the pain-points? What are the biggest impediments for them to interact with your precisely designed “colleague”? How to overcome the bias against chatbots that results from bad experience?
Read MoreSecurity Threats and Security Testing for Chatbots
This article is pointing out security threats and attack vectors of typical chatbot architectures — based on OWASP Top 10 and adversarial attacks .
Read MoreThe ultimate Guide to a Botium Box Proof of Concept (PoC)
Does Botium Box meet my requirements? Will it solve my biggest challenges and pain points? Is it worth upgrading my free Botium Box Mini? And anyway, which plan would be the right one for me? Let me take you on a quick journey through a Botium Box Proof of Concept (PoC) to answer those questions.
Read MoreRegister an Inbenta Chatbot to Botium Box
Our comprehensive test suite has been expanded with another connector so that you can take advantage of the benefits offered by Botium even if you are developing your chatbot with Inbenta. The following short blog post shows you how to connect your Inbenta chatbot to Botium Box in 2 easy steps!
Read MoreHow to improve your Teneo chatbot by test automation with Botium
We’d like to believe that we have made an impact on the conversational AI market with our long valued partner, Artificial Solutions and their platform Teneo, by supporting each other to perfect the development and the testing of all chatbots out there!
Read MoreDetecting problems of a chatbot and fixing them: OpenAI GPT-3 tricks and tips
All chatbot testers are dreaming of two buttons. One for detecting all problems of a chatbot, and another one for fixing them all. With OpenAI we were able to add some nice features to Botium Box which are going in that direction.
Read MoreHow to maintain chatbot regression tests with minimum effort
The biggest and the most hateful challenge in software development is writing test cases and maintaining them. This is no different when it comes to chatbot development. At Botium we don’t write the regression tests, we generate them. This article shows you how we do this with the least amount of effort invested.
Read MoreTesting Chatbots for the Unexpected
How is It possible to test something for all possible unexpected user behaviour in the future? While we do not own a magic crystal ball to look into future usage scenarios, from our experience we gained the best results with a systematic approach in a continuous feedback setup.
Read MoreThe 5 Step Journey to Becoming a Chatbot Tester
“What’s the best way to start testing a chatbot ?” The answer is surprisingly simple: Build up knowledge by educating yourself, and you will be able to answer this question yourself. This post will walk you through the major milestones that test automation engineers have to take for building this knowledge.
Read MoreDetecting dead ends in the Chatbot Conversation Flow
Botium Crawler is the newest member of the Botium toolset. It was designed to imitate users going through all the possible combinations of the dialog structure at the same time. It automatically detects all conversation paths for navigating through the whole conversation model.
Read More3 Steps: Register Oracle Digital Assistant in Botium Box
This article shows how to setup an Oracle Digital Assistant (ODA) chatbot in Botium Box.
Read MoreHow to create & build your perfect chatbot
The first thing we need to know: There is no such thing as the perfect chatbot! Even in the creation of a chatbot, there is no best practice guide. Every chatbot behaves differently, depending on the purpose, topic coverage and the target group. In general, however, there are various ways to build a chatbot and improve the quality in order to achieve a better conversion rate. In the following article we discuss the most important facts to build your “perfect” chatbot.
Read MoreUpskilling Test Engineers for Chatbot Projects
Test engineers learned how to test websites with Selenium and smartphone apps with Appium in the past, and now they try to apply this valueable knowledge again — neglecting the fact that chatbots are a new kind of apps that require new kind of tools (like Botium).
Read MoreTesting SMS Chatbots with Botium Box
Botium Box offers two methods to test an SMS Chatbot. It can act as a user sending SMS messages, or can interact with the API behind the chatbot. Sending SMS messages looks like an optimal choice because it covers all the stack, but it has a major drawback, the increased costs.
Read More4 DO’s and 3 DON’Ts for Training a Chatbot NLP Model
A quick summary of 7 important DO’s and DON’Ts when training an NLP model for a chatbot. They are best applied before starting a project, but can also help to build a mindset for quality training data in all chatbot project phases.
Read MoreGiving a Voice to SAP Conversational AI
Voice platforms like Alexa and Google Assistant make it easy to provide a custom voice experience to your clients, even without going deeper in audio processing — everything is part of the platform. But what if you already invested quite some effort into building a chatbot on SAP Conversational AI ? You certainly don’t want to switch to a totally new platform now.
Read More4 DO’s and 3 DON’Ts for Chatbot Testing Strategies
A quick summary of 7 important DO’s and DON’Ts when designing a chatbot testing strategy. We are continuously seeing teams ignoring those actually rather simple rules.
Read MoreBeginners Guide to Automated Voice App Testing
This guide suggests best practices, infrastructure and tools to ensure your voice app continues to deliver outstanding user experience.
Read MoreUnveiling the Value of Chatbot Testing
“What is the value of testing chatbots?” — A question that hits right in the heart of every passionate tester. This article explains the value of chatbot testing and the benefits you gain through AQ.
Read MoreGiving a Voice to Rasa with Botium Speech
Voice platforms like Alexa and Google Assistant make it easy to build your own voice experience, even without going deeper in audio processing — everything is part of the platform. But what if you want to rather go for a solution hosted by yourself, running an assistant on your own website, in your own infrastructure ?
Read MoreGive your Tests a Voice – Botium Box 2.8.4 is out
Botium Box 2.8.4 is out now - and it comes with all you need to test your voice applications - audio recorder, voice synthesis, audio effects. And more.
Read MoreHow to Automate Testing of your WhatsApp Chatbot
Botium delivers again — in this case, automated end-2-end testing of WhatsApp chatbots on real or virtual devices. For the first time it is now possible to have a full enterprise-level test strategy for WhatsApp chatbots.
Read MoreQuality Metrics for NLU/Chatbot Training Data / Part 2: Embeddings
Getting confused with terms like embeddings, utterance similarity, intent separation and cohesion? Gedankenblase Get educated on NLU quality metrics now.
Read MoreAssessing Quality of Chatbot Training Phrases for Watson, Dialogflow and everything else
This article shows you how to analyze and evaluate the quality of the training phrases for your chatbot intents with Botium. The purpose of this analysis is to avoid confusing the agent with phrases irrelevant to the intents supplied to, or more relevant to, other intents.
Read MoreAutomated Test Case Recorder – Introducing Botium Crawler
So you decided to add automated testing to your chatbot project ? Good decision 👍! But don't underestimate the effort 😓 in composing a test suite with high test coverage from scratch. Botium Crawler to the rescue.
Read MoreAutomated Testing and Monitoring for Alexa Skills with Botium Box
Botium Box and the Alexa Voice Service (AVS) connector can perform automated testing for Alexa skills – no speaking out loud necessary.
Read MoreTesting a Chatbot End-to-End with Perfecto and Botium
This tutorial provides step-by-step instructions how to setup Botium Box with Perfecto Mobile and Perfecto Web to run continuous End-To-End tests for a chatbot published on a website.
Read MoreTutorial: Analyze and Improve Performance of IBM Watson Assistant Skill
If you have an existing Watson Assistant skill and you want to analyze it for consistency as well as performance of the training data itself, you should read this article. It shows ways to improve the performance of your skill as well.
Read MoreVideo Tutorial: Improve Chatbot Understanding with Botium Coach
This video tutorial shows how to use Botium Box and Botium Coach to evaluate the performance of an NLU model trained with Rasa.
Read MoreUsing Google Sheets with Botium Box
Did you know that it is possible to connect your Google Sheet to Botium Box?
Read MoreTutorial: Run Selenium Grid and Connect to Botium Box
A Selenium Grid is a set of connected Selenium nodes each running your Selenium tests on a different kind of operating system on a different kind of web browser in a different version. We are using docker-selenium to boot our own Selenium Grid on our own server.
Read MoreTutorial: Using Paraphrasing to Increase Conversational AI Test Coverage
This tutorial provides a step-by-step introduction how to use Botium Box to quickly build a regression test suite for an Amazon Lex chatbot and increase the test coverage by adding paraphrases to the test data.
Read MoreBuilding a Chatbot Doesn’t Make You an A.I. Company
According to the report The State of AI 2019: Divergence there is a lot of Bullshit Bingo around the thing called Artificial Intelligence (AI).
Read MoreTutorial: Building a CI Pipeline for Chatbot Developers with Rasa X and Botium Box
This tutorial shows a way to establish the same methods to chatbot projects, especially for projects focused on training an NLU engine and/or a dialogue engine.
Read MoreFull-Blown Open Source Speech Processing Server Available on Github
Botium Speech Processing combines the best Open Source speech processing tools in a single service and makes them accessible with a HTTP/JSON API.
Read MoreTutorial: Install Botium Box on AWS (or any other cloud service)
Quick installation instructions for running Botium Box (all flavours) on a bare bone AWS Linux server. Same principles apply for all other Linux servers.
Read MoreTutorial: Benchmark Your Chatbot on Watson, Dialogflow, Wit.ai and more
Ever wanted to know if your company chatbot would work better on IBM Watson than on Dialogflow ? Or should you give Wit.ai or Amazon Lex a try ? With the help of Botium you can easily benchmark and compare the performance of established chatbot platforms with your own data.
Read MoreQuality Metrics for NLU/Chatbot Training Data
What is a Confusion Matrix ? How to generate and read a Confusion Matrix ? How to calculate precision, recall and F1-Score for your NLU engine ?
Read MoreBotium in a Nutshell, Part 9: Botium Test Levels or Selecting a Botium Connector
This is part 9 of the Botium in a Nutshell series of articles.
Read MoreTutorial: First Steps in Botium Box
So, you followed the installation instructions and you can see the Botium Dashboard, but you don’t know how to start ? Follow the steps in this tutorial to get your first Botium Box test project up and running!
Read MoreAdd Botium Test Suite to Jenkins Pipeline
One of the questions we hear the most is how to integrate Botium Box with Jenkins.
Read MoreBOTIUM BOX 1.7 is out now
We are super excited to announce that Botium Box 1.7 is out now! Discover great new features like Continuous Security Testing based on OWASP ZAP or our brand new Testcase Generator.
Read MoreBotium in a Nutshell, Part 7: CI/CD Pipeline Integration
This is part 7 of the Botium in a Nutshell series of articles.
Read MoreBotium in a Nutshell, Part 8: Best Practice for Test Case Development
This is part 8 of the Botium in a Nutshell series of articles.
Read More“Let’s build a Chatbot!” — How you get started
Chatbots are cool. They are driving the industry like hardly any other topic these days and are considered to be real game changers.
Read MoreBotium in a Nutshell, Part 6: E2E-Testing with Botium Box
This is part 6 of the Botium in a Nutshell series of articles.
Read MoreRegression Testing an NLP model with Microsoft LUIS and Botium
In the age of agile development, the age of manual regression testing is over. We developed a Botium extension to enable an agile developer team to run automated regression tests on a Microsoft LUIS model within their CI Pipeline.
Read More“Ok Google, how are you ?” — Testing Google Actions with Botium
Testing a Google Action by talking to your device isn’t fun. In the age of agile development, the age of manual regression testing is over. Within 3 days, we developed a Botium extension to enable an agile developer team to run regression tests for a Google Action within their CI Pipeline.
Read MoreBotium in a Nutshell, Part 5: The Botium Box
This is part 5 of the Botium in a Nutshell series of articles.
Read MoreChatbot Testing Done Right: Botium Box Available Now
Chatbots are driving the industry. With Botium we are driving chatbots. After months of hard work we are finally there: we are proud to announce the general availability of the newest Botium Stack member, the Botium Box.
Read MoreTesting Alexa Skills with AVS, Mocha and Botium
Testing an Alexa Skill by talking to your device isn’t fun. Some months ago I published an article showing several options for automating testing of your Alexa Skill:
Read MoreBotium in a Nutshell, Part 1: Overview
This is part 1 of the Botium in a Nutshell series of articles.
Read MoreBotium in a Nutshell, Part 2: Writing Chatbot Tests
This is part 2 of the Botium in a Nutshell series of articles.
Read MoreBotium in a Nutshell, Part 3: Automating Chatbot Tests
This is part 3 of the Botium in a Nutshell series of articles.
Read MoreBotium in a Nutshell, Part 4: Advanced Topics
This is part 4 of the Botium in a Nutshell series of articles.
Read MoreChatbot Training: Quest For the Holy Grail
It’s easy to make a chatbot fail. I am not talking about nonsense input, or input not related to the chatbot domain — literally all state-of-the-art chatbots will fail in these cases. No, I am talking about input making perfectly sense in the domain of the chatbot — simple and short instructions which should trigger one of the main use cases for the chatbot.
Read MoreHow I Used Tosca to Test an Alexa Skill
Today I was asked to build a proof-of-concept for using Tricentis Tosca to test an Alexa Skill — challenge accepted!
Read More