Best practice is to schedule at least two performance load tests with sufficient time between them to correct defects, though complex implementations may require a third test. This approach ensures you uncover hidden issues before they impact your customers in production.
What is performance load testing?
Performance load testing is a method of validating that your contact center infrastructure, including telephony components, IVR systems, and routing configurations, can handle expected call volumes without failure or degradation. It simulates real-world traffic to identify defects before they reach production.
Why multiple load tests are essential
This scenario may sound familiar: You’ve just upgraded your contact center, but before you can celebrate, there’s one last item on the project manager’s implementation plan, a performance load test.
Before you move the applications to production and schedule the actual cutover, you need to be completely confident that everything is working as expected. Your customer experience scores depend on it.
Often a project manager schedules just one test, but it’s actually best practice to schedule two performance load tests with enough time between the tests to correct any defects you find. Here’s why: you may have modified code in one part of the application, but your changes can cause issues in other parts of the application that you didn’t even touch. Also, the call flow from the ingress point to arriving at an agent’s telephone with associated screen-pop data traverses many components in its journey, and the complexity will have hidden defects. These defects will become exposed in the production system. You can’t predict in advance what these issues will be or where they will occur.
As a solutions engineer at Cyara with over eight years of experience supporting enterprise contact center implementations, I’m frequently involved in performance load tests for our customers. Before sitting down to write this blog post, I made a list of the different kinds of issues our Professional Services team has uncovered in various performance load tests over the past 8 years. The list included 45 items, which is much too long to include in this post, so here are the 10 most common:
1. Telco problems – SIP trunk misconfigurations or capacity limitations
2. Call connect time delays and disconnects – Infrastructure unable to establish connections within acceptable thresholds
3. Busy tones – Insufficient trunk capacity causing call rejection
4. IVR prompts incorrect – Wrong audio files or logic errors in prompt delivery
5. Calls incorrectly routed – ACD or IVR routing logic failures under load
6. IVR application issues – Application crashes or timeouts during high concurrency
7. Garbled audio – Codec or network quality degradation under load
8. Dead-end call flows – Missing error handling leaving callers stranded
9. Screen-pop failures – CTI data not reaching agent desktops correctly
10. Database related delays – Backend queries causing IVR response latency
Recommended load testing workflow
Follow this step-by-step process to ensure comprehensive load testing:
1. Schedule the first load test – Run initial test at full expected capacity.
2. Identify defects – Document all issues uncovered during testing.
3. Correct issues – Work with vendors and internal teams to resolve defects.
4. Run verification test – Execute second load test to confirm fixes.
5. Assess results – Determine if additional testing is needed.
6. Run third test if required – Address any new issues discovered in step 4.
A typical load test scenario
The first performance load test usually uncovers an initial “punch list” of defects and performance issues. Remember that call flows are complex and traverse across many components where defects are hidden. Once these issues are checked off as corrected and your QA testing team is confident that everything is in order, you can run the second test to verify that all systems are go. But sometimes surprises occur, and you may need to run a third performance load test.
Remember, it’s not about how many times you have to run the test, it’s about making sure that you find the issues that end up wrecking your customer experience before your customers do. Defects are significantly cheaper to fix before they end up in the production system.
Customer example: 3 tests to success
Recently, a customer scheduled two four-hour performance load tests with the following specifications:
• Concurrent calls: 4,000
• Virtual agents: 3,000+
• Test duration: 4 hours each
• Call rate: 7 CAPS (call attempts per second)
As it turned out, unexpected issues required a third test to be executed.
What each test revealed
Test Issues found Root cause Resolution
1st test System collapsed at 800 concurrent calls SBC and Voice Gateways couldn’t handle load Vendors provided software patches and configuration updates
2nd test Calls never exceeded 1,200 (should be 4,000) Telco SIP trunk tables incorrectly configured Telco corrected SIP trunk size tables
3rd test Call flows routing to wrong agents or dead ends ACD routing configuration errors Routing technician made corrections during test
Detailed test results
1st performance load test
Result: Found that the SBC and Voice Gateways collapsed under load as the concurrent calls ramped up to 4000 ports and issues happened at 800 calls. The production system would have collapsed within the first hour of the day before even getting to the peak busy hour if we hadn’t caught this defect.
2nd performance load test
Result: Found that the Telco SIP tables were incorrect and calls never exceeded 1200 calls when the trunk size was assigned to handle 4000 calls. This would have left 2300 agents sitting idle as calls would never arrive to them and customers would get fast busy signals.
3rd performance load test
Result: Found several call flows routing incorrectly to the wrong agents or arriving at a dead end. Customers and agents become frustrated when they arrive at the wrong queue, which requires an agent to transfer them to the correct queue, or if they arrive at a dead end with no agent intervention available.
As the project team reviewed the load test reports, they were able to confirm:
• Telco SIP trunk sizing and QOS values were correct.
• SBC and voice gateways were able to support the 4000 concurrent call load without failure.
• The IVR provided the correct prompts and could handle the call load while routing to the correct ACD queues.
• The ACD call flows routed the calls to the correct agents and agent groups and provided the correct screen-pop data.
• The ACD telephony infrastructure was able to support the 3500 concurrent agents receiving calls.
The next morning, the first email of the day from the project manager’s laptop had the simple statement, “The contact center platform and its applications are ready for production. All tests have passed!” That’s the message you want to see, no matter how many load tests it takes to get there.
Key takeaways
1. Schedule at least two load tests – Plan for a minimum of two tests with time between them to address defects.
2. Expect hidden defects – Complex call flows traverse multiple components where issues remain hidden until tested at scale.
3. Budget for a third test – Complex implementations often require additional testing rounds.
4. Fix defects before production – Issues are significantly less expensive to resolve during testing than after go-live.
Frequently Asked Questions
A single load test identifies defects, but fixing those issues can introduce new problems or reveal previously hidden ones. A second test verifies that corrections work correctly under load, and a third test may be needed for complex implementations.
Common issues include telco configuration problems, infrastructure capacity limitations, IVR application errors, incorrect call routing, audio quality degradation, and database-related delays.
Run a third test when the second test reveals new issues that weren’t present in the first test, or when fixes made after the second test are significant enough to warrant verification before production deployment.
Best practice is to schedule at least two performance load tests with enough time between them to identify and correct defects. However, unexpected issues may require a third or more tests, and the priority is finding all problems before they reach production.
Code changes in one part of an application can cause issues in other areas that were never modified, and the complexity of a call flow traversing many components means hidden defects are common. A single test rarely surfaces all of these issues at once.
Load tests commonly reveal problems such as telco issues, call connect delays, busy tones, incorrect IVR prompts, misrouted calls, garbled audio, dead-end call flows, and database-related delays.
Defects that reach production directly impact customer experience scores and are significantly more expensive to fix than those caught during testing. In one example, an uncaught SBC failure would have caused the entire contact center to collapse within the first hour of the day.
Based on the customer example in this article, four-hour tests are used to validate full contact center performance, allowing the project team to monitor real-time reporting and confirm all elements are working as expected throughout the test.
A thorough load test should confirm that telco SIP trunk sizing, session border controllers, voice gateways, IVR routing and prompts, ACD call flows, screen-pop data, and agent telephony infrastructure all perform correctly under the full expected call load.
A platform is ready for production when all scheduled load tests have passed and the project team has confirmed that every component, from ingress infrastructure to agent routing, is functioning as designed under the required concurrent call volume.
