Yes, there probably are lots of other issues you are concerned about diagnosing, debugging and tuning, but you might want to know just one more thing about a load test — how do you know when you pass the test?
There are no letter grades here, and unless something breaks under load (see our previous two posts, Is It Time for a Checkup? and Managing Your Worst-Case Scenario for examples), you might not be able to tell. You get a numeric score; for example 98.7%, and the details of your test, which probably look something like this:
Dialed 120,000 calls with a peak of 5,000 simultaneous calls. The max CAPS was 20, the average delay a customer would experience is 2.4 seconds, and 1.3% of calls got a mysterious message saying “Please call back later.”
This is great information — but how do you know if 98.7% is a passing or a failing grade? And, the answer is, “It depends.”
Define Your Target
Zig Ziglar, a world-famous motivational teacher and trainer, once said, “If you aim at nothing, you will hit it every time.” The point here is that passing or failing a load test depends on the target pass rate you set. You can’t know until you have defined a target to hit. This point is often overlooked, but it is key to every project and should be validated against a contract or any other standards that are defined for your contact center.
So, how do you define the target? Now, don’t be surprised if you get lots of answers when you start asking this question. A lot of people are likely to have opinions. However, be careful not to jump to a generic answer. Consider the specific needs of your business.
As a rule of thumb, here are some guidelines for setting targets based on common scenarios:
- Pure technology refresh projects: Your pass rate should be above your current measured baseline. This means measure your existing system before you replace it. Be careful, as this isn’t always the best approach. For example, if your existing system is consistently dropping calls, you will want to set the bar higher.
- Low-budget projects that are not mission-critical applications: 97% + may be a realistic target. Your Help Desk and Payroll are good examples of this type of application. Here you have a somewhat captive audience, and any callers are likely to call back if their calls are dropped. So, for these callers, it may be acceptable to have an audio hiccup or other minor issue from time to time.
- Revenue-impacting applications: 99% + is probably where you need to be, after all these calls are where you make your money. Again though, consider your business. If you are selling million-dollar mortgages, you may not be able to afford to drop even one customer in a hundred.
- “Carrier-grade” applications: 99.7% + is most likely what is defined in your service contract. That’s three-tenths of a point away from perfect and is pretty reliable.
- Emergency services: Here you have to be at 100%. There is no tolerance. After all, we are talking about possible dire occurrences, possibly even death, if you drop a call.
Other Factors to Consider
With a general target in mind, you can start to consider other factors that might influence your target pass rate. Some of these could be:
- Are there any contractual obligations to meet?
- How committed is your company to the customer experience? Let’s face it; some companies are more willing to invest in CX than other companies.
- What kinds of failures are acceptable? A dropped call can be considered a fail, but should you count calls with an extra ring splash or a 5-second delay as a failure?
- Is maximum load a stretch target or a daily occurrence? Or, should you set two pass rates – one for maximum load and one for everyday traffic?
- What’s the cost of implementing a higher target? How does this compare to the potential revenue lost or the cost of a service level agreement (SLA) breach?
- Can the project budget support your target?
Be Realistic
For most companies, a 100% pass rate isn’t a realistic target when pushing systems to their limits. The cost of over-engineering a platform to guarantee that level of performance is prohibitive.
You can save a lot of headaches by setting a realistic target that meets your business needs before doing a load test. A passing grade is a platform that delivers to your defined target pass rate.
Testing contact center performance can be a difficult task. Don’t make the mistake of focusing on the technology risks and ignoring your business objectives. When you are clear on the level of performance you need, it’s easy to know when your platform gets a passing or failing grade.
Find out more about Cyara Load Testing.
