The Hokey Pokey of Continuous Testing: You Put Your Shift-Left In, You Put Your Shift-Right In…
Testing what you’ve developed before the big, red “go live” button is pressed isn’t a new concept. It’s been a crucial part of software development for a long time, and a practice that development teams employ everywhere. When and how often that pre-deployment testing should occur is a more recent topic of discussion, with the “shift-left” label and plenty of proof points to back up its benefits.
In today’s post, we’re taking a look at the other “end” of the continuous testing (CT) DevOps loop and exploring the world of shift-right testing – or what we call CX monitoring – which involves testing in deployment.
Let’s start with a quick vocabulary lesson and review some definitions of these terms.
At a high-level, Continuous Testing can be defined as the processes of conducting automated tests as part of the software delivery pipeline, continuously gaining data and feedback about risks or bugs that could affect successful software releases so that development teams can address them as they are found. There are 14 key points about CT here that can better help you understand this concept.
Shift-Left Testing specifically refers to the practice of testing in the early stages of the software development life cycle. Done correctly, this reduces the number of bugs and increases the quality of your code so that during the deployment phase of your project, no code patches are required. It’s important to note that shift-left testing significantly lowers the cost of bug fixes compared to when or if you find and have to fix bugs in late-stage development or post-production.
Shift-Right Testing, as you can probably deduce from the two definitions above, is the concept of testing later in the development process, usually after deployment/release, and while in-production. If it sounds a little “too late,” shift-right testing encompasses deployment, configuration, operation, and CX monitoring, and isn’t something you do instead of shift-left and continuous testing, but in addition to it. It is arguably most important to ultimate success. Now, let’s talk about why.
Shift-Right Testing is how you can realize and extend the benefits from adopting continuous testing. By continuing to test your systems (including things like integrations, features, and call flows) right into your live CX environment, you are closing the loop on your development cycle and testing at every stage for better overall outcomes and faster results.
It takes work. Your development team will need to learn new skills and utilize technology platforms and solutions that allow them to monitor, aggregate, and examine production data that point them where to find and fix issues, but studies show that shift-right testing adopters are finding the time and effort investment well worth it. A survey conducted by Capgemini and Broadcom showed that CX testing in production was ranked as the number one practice currently implemented or planned by respondents.
Why the Shift to Shift-Right Testing?
As forward-thinking companies evolve their practices to adopt Agile practices, looking to move faster, more efficiently, with increased accuracy and improved output, it’s more and more important to work in both directions – left and right – within the development cycle. This requires testing and operations teams to work in unison.
This new approach is bringing to light a new discipline – DevTestOps, which goes beyond DevOps and drives teams to test in both left and right directions for 3 major reasons:
- Without engaging with your live systems by testing in production, it’s impossible to accurately measure the success (or shortcomings) of your customer experience.
- Things break. It’s natural and expected that, even if going into production your systems has passed their tests with flying colors, there’s no guarantee they will remain bug-free forever. Your live environment may be slightly different than your development or testing environment, or a manual tweak to a live system could change things in an instant. Vigilant and quality-focused organizations know they must continue to test on the right side of go-live.
- A key principle of DevOps is that 100% reliability is unrealistic (and expensive). There’s a great blog post on this topic available here, delving deeper into this concept.
So how can you best adopt continuous testing by adding robust shift-right testing to your current development practices? Here are a few key things to focus on to make this a smooth transition:
- Prioritize Automation: Monitoring all your customer journeys and interaction channels by repeatedly testing your systems after deployment will take a lot of work and manpower if you have to do this manually. By automating test case creation, execution, and reporting, you’ll save time and be able to reallocate your team members to higher-value projects.
- Collaborate: By supporting relationships between developers, testers, operations teams, and business teams, you’ll be able to work together to achieve a better overall outcome for your organization. This is where the DevTestOps team comes together.
- Collect and Share Results: Testing at any stage of the development life cycle – before, during, or after deployment – isn’t valuable unless you use the results to make improvements. You must find a way to easily collect, organize, display, and share the results in a way that shows exactly where to go to fix issues, improve quality, or add features and innovations.
- Leverage CT Solutions: Finding platforms that allow your teams to work from a central hub to execute tests, review data, report results, and make changes will be key to your success. To help our customers do this, we’ve designed Cyara’s Automated CX Assurance Platform to support the entire software development life cycle for customer experience, from design to functional and regression testing, user acceptance testing, and load testing, with CX monitoring baked right into the platform. Users can leverage our web portal to monitor customer journeys in production, and utilize clear, graphical dashboards and extensive sharing capabilities to report on performance to various business units and teams, enabling better collaboration, transparency, and faster issue resolution.