This article was originally published on Botium’s blog on January 28, 2021, prior to Cyara’s acquisition of Botium. Learn more about Cyara + Botium
When it comes to testing WhatsApp chatbots up to now there have been mainly two approaches:
- Testing manually on a smartphone
- Testing backend functionality with API Testing
Automated vs Manual Testing
Both approaches are valid and no enterprise-level test strategy should miss any of them. But there are two obvious flaws:
- Manual testing is not scalable to the extent that is required to guarantee constant high quality for production-level chatbots
- Backend testing does not consider the end-user experience
Botium fills this gap with a new connector for testing WhatsApp chatbots on real and virtual smartphone devices.
Building Test Infrastructure
Here is what you need:
- Some smartphones with SIM cards and USB cables
- Or Docker + Docker-Compose to launch virtual devices
Setting Up Real Devices
- Go and purchase some smartphones (or use your own one) with SIM cards
- Install and register WhatsApp on them
- Enable Android developer mode — ask Google how to do it for your device model
- Attach it to your workstation with the USB data cable
- Install Android SDK
- Install and launch Appium
Setting Up Virtual Devices
- Install Docker and Docker-Compose
- Launch a virtual device with the help of this project
In short, you can create a file docker-compose.yml and launch it with docker-compose up -d to get a single virtual machine with a single virtual Samsung device up and running. You can see it in action by browsing http://localhost:6080
version: '3'
services: null
samsung_galaxy_S8: null
image: budtmo/docker-android-x86-11.0
privileged: true
ports:
- '6080:6080'
- '4723:4723'
- '5555:5555'
environment:
- DEVICE=Samsung Galaxy S8
- APPIUM=true
- MOBILE_WEB_TEST=false
- AUTO_RECORD=false
Setup and Configure Whatsapp
Now you have to install Whatsapp on this virtual device. There are several options for this, here is the way I did it:
- Download the Whatsapp APK here
- Install the Android SDK on your local machine, you will need adb
- Connect to the virtual device:
adb connect localhost:5555 - Upload Whatsapp APK to the virtual device:
adb push /path-to/WhatsApp.apk /data/local/tmp/wa.apk - Install the Whatsapp APK:
adb shell pm install -r /data/local/tmp/wa.apk - Launch Whatsapp:
adb shell monkey -p com.whatsapp 1
Now you have to register WhatsApp with a phone number of your choice — purchase a SIM card or use a service like Twilio.
In one of the next Botium releases, the registration step will be automated by Botium Box as well.
Additionally, you have to register the Whatsapp Chatbot you want to test as a Whatsapp contact. The name of the contact is required when configuring Botium later.
Do any tasks regarding registration, terms&conditions etc required by your Whatsapp Chatbot upfront, as this things typically are not part of the test cases.
Now that your devices are up and running and WhatsApp is available, let’s see what else to prepare.
Connect Botium to WhatsApp
You need an installation of Botium Box for this purpose. Experienced DevOps engineers can give Botium Core a try, the free and open-source automation library powering Botium Box and other Botium products.
Get your copy of Botium Core here
Connect Device Lab To Botium
In the Botium Box settings, register a new device provider.
Tell Botium about the Device
While for most device cloud providers the available devices can be listed automatically (by calling the device cloud listing APIs), this is not possible for your local Appium installation. Edit the file LOCALSELENIUM.json in the resources folder of Botium Box to tell Botium about the available devices:
[
{
"name":"Samsung Galaxy S8 Emulator",
"value":{
"type":"MOBILEAPP",
"capabilities":{
"appium:platformName":"Android"
}
}
}
]Compose a Device Set
Botium Box groups the devices you want to run your tests on in device sets. Create a new device set for your Appium endpoint and select the Samsung Galaxy S8 Emulator (and maybe other devices as well if you connected them).
Configure Botium Connector for Appium
Register a new chatbot in Botium Box
- As Connector/Chatbot Technology use WebdriverIO (Selenium or Appium)
- As Automation Technology select Use Appium
- As Webdriver Script choose Whatsapp
As an experienced Appium developer you might ask Where do I enter the Selenium CSS Selectors ? — with Botium you don’t have to do this as this is part of the Webdriver Script Whatsapp
On the last step of the Quickstart Wizard, make sure to select your device set at the very bottom to start your tests.
Write Test Cases
You can now use the full power of BotiumScript to write your test cases. An easy one could look like this:
hi#me
hi#bot
Welcome to the World Health OrganizationWatch Botium Automating Whatsapp
When running a test case, Botium will now
- open the WhatsApp app
- select the contact representing the chatbot
- cleanup the message history
- send test case input from WhatsApp
- receive WhatsApp output and compare with the test case
- repeat the last two steps until the test case is ready
Wrap-Up
With Botium Box and Appium, it is now possible to run automated end-2-end tests of the conversational flow of your WhatsApp chatbot.
