Test Automation candidates strategy — easy steps to achieve maximum efficiency

A key factor for success in test selection for automation is to proactively assess the effort needed to uphold test stability. What's the purpose of executing tests that consistently result in failure? While the goal in QA is often to achieve 100% test automation, it’s essential to remember that quantity doesn’t necessarily equate to quality. Striving for 100% efficient test automation adds more value. To maximize the potential of your test suite first build your Automation candidate strategy. Once the strategy is in place, it will help to control the test suite to meet several key criteria:

• Relevance — ensure that the tests in your suite are relevant to the current state of the application and its critical functionalities. Eliminate or update tests that are no longer applicable.
• Coverage — achieve comprehensive coverage of your application’s functionalities, including core features, edge cases, and common user scenarios.
• Stability — develop tests that are stable and resistant to frequent failures. Minimize false positives and negatives by maintaining consistent test environments and data.
• Maintenance Efficiency — create tests that require minimal effort to maintain and update, even as the application evolves. Implement practices such as data-driven testing and parameterization to ease maintenance.
• Execution Speed — optimize test scripts for fast execution to reduce testing time. Utilize parallel testing to further speed up the process.

So how do I start an assessment of the best candidates? How can I make sure the strategy leads toward test automation success? As an option you can follow one of the approach that is commonly used:

Two primary dimensions should guide your decision-making process: Effort and Impact. When assessing a test scenario, consider asking the following questions: “Is it relatively easy to automate, and what is the effort required to incorporate it into the framework?” Another critical question to aid in your decision-making is, “What impact will it have, particularly in terms of coverage for the most critical modules and flows?”

This type of assessment will produce four categories of test automation candidates:

• Quick wins — the easiest effort but the least impact. This category can be included into test automation as second priority and when time / resources allow.
• Best candidates — relatively easy effort but the most impact. This category is the best to start automate as time /resources investments is payed back with good coverage.
• Worst candidates — difficult to automate and does not bring much of a value. This category is better to exclude from test automation strategy as potentially it will only demand time/resource spend on the maintenance such test cases with no or very low probability of finding regression defects.
• Game changing — difficult effort but high impact on test coverage. These type of scenarios most probably is time consuming to execute manual as well so you should expect that test maintenance also will be difficult. However as a long term run it may pay back with consistent quality in the most crucial functional flows. When you build an automation strategy consider such candidates to be added to coverage in incremental way e.g. 5% at the beginning of system testing, then increased to agree % closure to production stage.

Test automation pyramid can help to finalize the candidates. Ideally automated test suite should be automated and triggered automatically with every new build to upper environment (for example run nightly on schedule or run on demand via CI/CD pipeline). Make sure based on type of execution you plan for required tools and resources. Here some help in making decision on what automation strategy to build:

Building a solid and stable technology ground to resolve business needs is very important. That’s why technology facing test cases usually are more preferable for test automation as they more isolated, they run faster, and less costly — required less time and resources to develop and maintain the scripts. They may fall into category “best candidates” or “quick wins” and ensure early feedback on the question “are we building the product right?”. More deep into technology more faster ROI (return on investment). At the same time business facing test cases need to be automated at some % as well so we can get early feedback on question “are we building the right product?”. These level test cases may fall into categories of “worst candidate” and will be part ofr you manuall testing strategy rather than invest to automate them, or could be part of “game changing” bucket and provide right level of system quality. This approach is helping preventing bugs and lower down fixes cost as we can catch and resolve way before production.

To get familiar with various tools and approaches on how to approach automation in your project please read “X as a Code” engineering strategy”.

If you are looking for automation of the regression suite you can compliment your approach with “Regression strategy in one go”.

Here is some example of “not a rule but most commonly followed approach” what could be considered as “best” and “worst” candidates:

In summary, a test automation strategy provides a structured approach to automation that ensures efficiency, consistency, and alignment with project goals, ultimately leading to improved software quality and faster delivery times. It’s essential to carefully evaluate the suitability of each test scenario for automation and prioritize based on factors like stability, repeatability, and the potential return on investment (ROI) in terms of time and effort saved through automation.