Efficiency is doing things right; effectiveness is doing the right things. -Peter F. Drucker
Automation in the software industry has quickly emerged from being “sophisticated and cutting-edge” to “mainstream and necessary”. Imagine a scenario where there is a new release and bug fixes in a working module. So if you have to ensure that the new fixes did not introduce any new bugs in the previously working module, what will you do? Will you just go and test each module manually (and imagine it’s a big website with hundreds of modules-as it happens in real world) with putting in hundreds of hours of manual effort. And will you do this every time there is a new release? Well, you might do it manually, but then the concept of “effective testing” is lost in the way, effective in terms of company cost, resources, Time, effort etc. Here comes the need for Automation.
However, bringing in the automation trend into a company’s regular workings does not come without its cost. Introducing automation requires additional investment in talent acquisition, frameworks and automation tools. And even with everything in place, poorly written automation scripts can cause the company unnecessary hassle and chaos and will require a lot of human intervention. Writing and running an improper script is equivalent to not running a script at all.
Measuring the success of automation is hence crucial in understanding if the efforts put into automation have been fruitful or need a revisit.
Automation success can be boiled down to the satisfaction of three criteria: More Efficient, More Effective, and Less Manual Effort. The mantra is to “Start with the end in mind!” To understand what we need to achieve!
With time being our primary resource of value, efficiency refers to any direct or indirect reduction in wastage of it. Effectiveness refers to how successfully defects are identified by our automation scripts. Less manual effort refers to any direct or indirect reduction of workload handled by human testers.