Continuous performance testing adheres to agile methodology, meaning speed and fluidity are the keys to success. You will find no shortage of articles detailing exhaustive lists of best practices to achieve this, no two of which will exactly agree on which practices make the cut.
Let’s instead identify the best of the best practices chosen because (a) they are so critical that you cannot get off the runway without them and (b) each one feeds into and enables the others. Take note of the inevitable overlap as you read on.
What is Continuous Performance Testing?
Continuous performance testing is the practice of tracking how an application performs under increased load on a continuous basis. Unlike preparing for peak traffic events, this practice incorporates performance testing into the CI/CD pipeline.
Continuous Performance Testing Best Practices
Here are five essential continuous performance testing best practices to consider:
- Focus on SLAs
- Be Test-Driven
- Test Early, Test Often
- Keep Communication Open
- Continuously Monitor and Report
Let us take a closer look at each one.
Focus on SLAs
First things first: Set a foundation of expectations based on your performance service level agreements (SLAs). This is your North Star.
Your application’s SLAs should determine what level of performance qualifies as a successful test result. For example, your SLA will likely require your application to always meet or exceed a certain level of throughput at any given point in time, which should set a baseline defining what merits a successful test.
BlazeMeter can help, even at this early stage. Once you know the SLAs, use BlazeMeter’s Performance Testing UI to customize how much of a load your application will be under during a test. You can easily tweak how many virtual users will hit your application over a set duration, then alter that load at any point.
BlazeMeter’s UI will help you easily test at the minimum required load, higher loads for various situations, and even extreme loads for when you need to know how it will hold up during a high spike in user activity.
Once your test executes a few times to your satisfaction, consider using one of those runs and its resulting report to set your baseline. BlazeMeter’s Baseline Comparison tool makes this especially easy, granting you the ability to define a baseline straight from the UI.
Be Test-Driven
Your continuous performance testing approach must be test-driven to be agile. Older models (such as the Waterfall Model) often saved their tester’s major work for a period nearer the end of the development cycle. A continuous, test-driven approach, however, requires testing throughout the entire development cycle – beginning to end.
Create new tests every time the dev team creates a new feature. You need not reinvent the wheel with every change; rather, your test suite should evolve concurrently with development so that you don’t have to recreate scripts from the ground up for every code change. Design dynamic, flexible tests instead.
BlazeMeter provides the capabilities to help. First, you can use the Performance Test UI to customize your own Scenario Definition. You can even upload your own scripts and any files associated with them.
As new code changes and features roll out, you can easily adapt by altering the scenario, replacing the script, or replacing the data files. You can also duplicate the test so as to have multiple versions and variations at the ready.
Second, leverage BlazeMeter’s Test Data feature to make your test even more adaptable. Once you upload a JMeter or Taurus script, you can upload a CSV of static test data or BlazeMeter can help you generate synthetic, dynamic data. You can easily reuse this data for countless other test scenarios. You can even connect BlazeMeter to a test data manager (TDM) server.
You can even start performance testing before you have any actual infrastructure to test with. BlazeMeter’s Mock Services let you create a software representation of infrastructure so that you can reliably simulate an environment before your real one even exists.
Test Early, Test Often
A test-driven model not only necessitates testing early, but it also means testing often – very often.
Testing should begin as soon as the earliest development code can possibly be executed. Every iteration that follows should kick off another round of tests, made easier and less time-consuming via the previously mentioned dynamic test configuration.
Once you’re ready to execute your tests, BlazeMeter’s scheduling feature can ensure tests are executed at the right time, automatically, so you don’t have to be at your keyboard.
Lastly, chances are you leverage various tools for your continuous integration (CI) pipeline or continuous delivery (CD). BlazeMeter makes this easy via integration options with many CI/CD solutions, such as Jenkins, TeamCity, AWS CodePipeline, and others.
Keep Communication Open
Frequent, transparent communication is a must at all times. Not only is this true of continuous testing; it’s true of any agile process. Yet we’ve all been there – An email never gets a reply or a phone call is lost to voicemail purgatory. That can’t be true here. Continuous testing absolutely depends on fluid, cross-functional collaboration; therefore, communication bottlenecks will sabotage the effort.
Developers and testers must work closely during every stage of the development cycle, from the earliest executable code to the final release, so that both parties share the same expectations for how the app is intended to function and how the app will be tested. In fact, testers should be included in scrum meetings.
BlazeMeter provides a comprehensive platform to empower collaboration, most of all with the ability to share reports. Each tester in BlazeMeter can easily share a test report with any colleague across any function at any time.
Continuously Monitor and Report
To put a twist on an old saying: If a test executes alone in a lab, and no one else is around to see it, does it yield any result?
Since you’re testing early, often, and in collaboration with developers, test reports should be passing hands every step of the way. All tests at all stages should be monitored with reports at the ready for whoever needs it, whenever they need it.
BlazeMeter’s dashboard provides a wealth of reporting options. Real-time reports show live results while your test is executing, culminating in a final post-test report rich with analytics.
Meanwhile, you can practice shift-right performance monitoring by integrating BlazeMeter with your favorite application performance monitoring (APM) tools.
Bottom Line
Achieving continuous performance testing is within reach by combining these best practices with the right continuous testing platform.
Source: https://www.blazemeter.com/blog/continuous-performance-testing
