RIGHT Blogs                                                              

Interactive Voice Response Systems (IVRS) Test Automation

It is challenging to setup automation for IVR systems because of the complexity of setting up test systems, Synchronization of calls generated with call flow being used, the ability of the automation framework to simulate all supported actions of the call flow and large volume of calls to be generated.

I wanted to share my experience in setting up test environments for automation testing when testing IVR systems with DTMF functionality. At the end of this blog, I have outlined how the same approach can be used for automated testing of Speech enabled IVR systems.

This approach requires two identical systems to be setup for automated testing of IVR systems. Call flows running on two systems will be different.

Let us name system under test as SUT. The SUT will be setup with various call flows to test different features/nodes of the application/product.

Let us name second system used for automated testing as AutomationSystem which is connected to SUT in back to back configuration. The call flow running on AutomationSystem must be designed / configured in such a manner that it can validate all possible nodes and flows running on SUT.

A Few tricks while designing the call flow on AutomationSystem are:

• 1. Call flow on AutomationSystem should start recording at nodes where SUT is expected to playback a prompt. This way the AutomationSystem is simulating an end user listing to the IVR system prompt. AutomationSystem should be configured to terminate recording on silence so that when SUT stops playing the prompt, AutomationSystem knows and moves to next node in the call flow.
• 2. Call flow on AutomationSystem should play out DTMF tones where SUT expects a response from the user. This will be simulate the end user, who will provide a response by pressing digits on his/her phone for IVR system to proceed with the call flow.
• 3. Call flow on AutomationSystem should also randomly simulate dial-ahead feature if the IVR system supports this feature. This can be simulated by waiting for a random number of seconds (less than the length of the prompt played by SUT) and dial DTMF tones.

The SUT should store the call flow details like nodes, prompts played, and DTMF received etc. in a database. The AutomationSystem should also store the nodes, recording files, and DTMF generated etc. in a database.

At the end of the tests, both tables can be joined to find the path traversed by the calls on both systems to validate if SUT responded correctly to all request from the AutomationSystem.

One of the major issues with an IVR system testing is how to stress test the system for stability. This approach can also be used to configure and execute stress test where the focus is not on functional testing but stability of the system.

For testing a speech enabled IVR System, the trick is to replace recording with speech recognition. Most IVR systems support speaker independent speech and if we are using a Text to Speech engine (TTS engine) for generating prompts, the accuracy of speech recognition engine will be almost 95%+. This way, we can use a TTS engine for prompt generation on SUT and speech recognition for recognizing prompt on AutomationSystem to validate if the correct prompt has been played and then provide a response intelligently.

I would like to hear more from others with experience in the field of IVR and speech enabled systems on how they have achieved automation without using expensive COTS tools.

-Sandeep