Quality Assurance Maturity Assessment — a journey to Excellence
How often do you and your team find yourselves brainstorming ways to enhance quality? In my team, “quality improvement” is a frequent topic of discussion at nearly every other retrospective session as because testing is never truly sufficient. And we always determine the next steps thanks to active use of the Quality Assurance Maturity (QAM) Assessment framework, which I would like to share with you.
Definition
Quality Assurance Maturity (QAM) Assessment helps identify the gaps that required action to improve Quality Assurance (QA) team productivity and end product quality within shorted cycle. This journey towards QA Maturity involves structured phases that guide QA team or department from rudimentary practices to sophisticated, optimized processes.
Benefits
- Enables QA team to track the progress it is making in each area of Quality Assurance Principles.
- Helps increase QA team level successes and adherence to the best practices thanks to increased transparency and clear identification of gaps and improvement areas.
Deep dive
Let’s delve into the stages of QA Maturity and learn how to progress through them to achieve excellence. Below, you can find proposal how to define five levels that categorize the quality based on the total QA Maturity score or “% of Compliance” against possible maximum score. You can adjust these levels and scoring scale based on your project specifics or use it as provided below. Please note that this article focuses on the concept of measuring QA adherence to current best practices. It’s important to remember that this process will require ongoing review and adjustments, as QA is a dynamic field with continuous advancements and new developments. Later in the article, I will provide an example of how to measure the score.
Survey principals
Assessment Survey is a key artifact in this process and should cover all defined categories that will represent holistic view of desired results. Just to give you an idea below is an example of a completed survey for two Categories, showcasing a minimum of two quarters of assessment with current scores and the highest scores that were previously achieved for comparison. This allows you to observe whether each Category is progressing or declining. For complete version of Assessment Survey please connect with me in LinkedIn and I will share it with you.
Now let’s get into details of what principles QAM Assessment framework consists of:
Each QA team or identified stakeholder goes through the Assessment survey.
10 Quality Assurance Categories are proposed to be considered for an evaluation. This helps to understand existing Quality process and challenges. Please note these Categories can be adjusted per your team needs.
Each Category has a set of Assessment statements. Number of Assessment statements can vary from category to category based on your project specifics. Example below is “Category 1 — Requirement elaboration” has 3 Assessment statements:
Each Assessment statement is evaluated within the scoring scale, in my team we defined it from 1 to 5. Please note this can be also adjusted.
“Current score” is assigned to an each “Assessment statement” as commonly agreed during the meeting or as average number from combined surveys.
For each Category Total “Given scoring for the current quarter” and Total “Max scoring achieved before” are calculated as sum of Assessment statements scoring. “Max scoring achieved before” can be called as “Historical score” and it’s defined based on previously taken survey as max score achieved from before. So this data will be available only after the second assessment. The initial survey will provide a strong starting point for future improvements. From the second survey onward, you will be able to track progress using historical data from previous surveys.
For each Category calculate “% of Category current Compliance”. It’s nothing but calculate ratio of the “Current quarter total score” to “Max possible total score” for a given Category (see example below).
Let’s take as an example “Category 1 — Requirement elaboration”.
To calculate “Category Max possible total score” use formula:
- total score = 15 as it has 3 Assessment statements and 5 is the Max possible of scoring scale (refer “1.5 Assessment Scoring scale”).
To calculate “% of Category current Compliance” use this schema:
- Calculate the current QA Maturity total score 11 = 3 + 4 + 4.
- Calculate “% of Category current Compliance” — take “Given Category Total scoring” and divide to “Category Max possible total score”. This will provide you an assessment of what level of QA Maturity is Category as of today.
All 10 categories are contributing to “Total QA Maturity scoring”
The total QA Maturity score are calculated as sum up of all categories total scores.
- Current total QA Maturing score — sum up all Categories scores 161 = 11 + 20 + 17 + 12 + 9 + 18 + 21 + 22 + 12 + 19
“% of Compliance Total rate” is calculated against Max score. See formula and example below:
- Based on the total score or “% of Compliance” the current stage of QA Maturity is defined.
Process Steps
Quality Assurance Maturity assessment processes is essential for identifying what is going well and what needs an improvement. Below is a step-by-step guide that represents an incremental and iterative process that should occur at a regular, agreed-upon cadence. In my team, we perform this assessment every quarter.
Step #1 — Assess
Basically this step is all about gathering information, conduct the Assessment Survey that we discuss above and complete survey results analysis.
Survey results analysis
Analysis should be completed on Category level as well as overall scoring result.
- For each Category compare “Given scoring for the current quarter” against “Max scoring achieved before” to see what areas are progressing well or what areas need more attention.
- For overall result analysis document what is the current QA Maturity Level (refer “1.1 Stages of QA Maturity”).
Step #2 — Plan
Once results of survey is available team will be ready to execute planing for improvements and create roadmap of Desired changes in 30, 60 or 90 days.
Where to start:
- Place each Category Current state of “% of Compliance” under relevant QA Maturity Level matrix
- Add Desire state of “% of Compliance” for each Category.
- Below example provides you visualization of how summary of “Current vs Desired” state can look like:
- Define actionable and prioritized initiatives based on timelines of short, medium or long-term goals to indicate by when you would like to achieve them.
Step #3 — Execute
Execution phase may vary and should consider specifics of your project, your team skillsets and current deliverables goals. In this article I will just high level indicate activities and possible deliverables:
Step #4 — Measure success
Again, “Success” definition will be specific to your team however in general “Measure success” of QA Maturity framework execution is evident when there is a measurable improvement in QA metrics around end product, or standardized processes, or increased automation, or progressed end user satisfaction, or continuous improvement, or positive team development, and data-driven decision-making. By consistently achieving these indicators, organizations can ensure that their QA processes are robust, efficient, and capable of delivering high-quality software products.
Of course monitoring quarterly will give you high level understanding where your team is heading, but don’t limit only to this type of criteria, always build as much as possible detailed metrics of your team Success measure.
Summary
Measuring Quality Assurance Maturity is a critical step on the path to excellence, and here’s why it’s essential:
- Insightful Analysis: helps identify what team is doing well and where there are gaps or weaknesses.
- Standard Comparison: allows team to benchmark their processes against standards or best practices.
- Consistent Quality: leads to more consistent and higher quality deliverables, reducing defects and increasing reliability.
- Risk Management: allows for better risk identification, management and proactive mitigation strategies.
- Efficient Use of Resources: by identifying the most critical areas for improvement and assigning priorities what to do now what can wait, resources can be allocated more effectively, ensuring maximum return on investment.
- Building Trust: demonstrating a commitment to continuous improvement builds trust and confidence among all involved stakeholders, management, and development teams.
- Culture of Quality: fosters a culture of quality within the organization, encouraging teams to strive for excellence.
