4 Quality Metrics in Scrum We Use to Stay Productive

8 out of 10 teams fail to deliver projects with 5 hours required to make a decision (ScrumInc).

These numbers show how essential quality metrics in Scrum are.

The better you measure, the higher is the project success rate.

In this post, you’ll learn our 4 quality metrics in Scrum we use to measure quality and keep teams on track. Here’s the plan:

• 4 Scrum metrics to boost your team productivity
• Formulas on how to measure quality in project management
• Examples of sprint metrics charts and tables
• Review of Agile Scrum methodology.

4 Scrum Metrics to Stay Productive

Sprint Velocity

Sprint velocity is one of the most widespread Scrum metrics. It gives you a historic overview of how much value you have delivered at every sprint.

In terms of one sprint, it is the amount of work (story points) your developers can complete under ideal circumstances.

Why sprint velocity is a crucial metric?

With this knowledge, you can plan projects and predict how much work can be completed in the next sprint.

The more accurate you are with this estimate, the more realistic your plans will be.

How to measure quality in project management with sprint velocity?

Everything you need is just to sum the story points completed at all sprints and divide them by the number of sprints.

We should mention that all examples in this article will start from the 2nd sprint. Our Senior Delivery Manager, Tatiana Vichkanova says:

For example, let’s calculate sprint velocity based on the below data:

• Sprint 2: 4 user stories x 12 story points = 48
• Sprint 3: 5 user stories x 12 story points = 60
• Sprint 4: 6 user stories x 12 story points = 72
• Total: [Sprint 2] + [Sprint 3] + [Sprint 4] = 180

So, your average sprint velocity is 180 / 3 = 60.

This way, you can now predict the number of sprints left to get all work done.

If you have 180 story points remaining to be done, you can assume that your developers will need 3 sprints more to complete the project.

However, only an estimate can be reached if all other things are constant.

One of the most effective ways to track sprint velocity is to draw visuals charts.

There are many cloud-based tools that allow you to draw multiple charts and share them with a Scrum team.

Below you can see an example of a velocity chart created at Ascendix Tech:

Sprint Velocity | Scrum Sprint Metrics

This chart allows you to analyze the amount of work done during each sprint to get an overall picture of the project activity.

Effort Estimation Variance

Over- and underestimations have a huge impact on a team’s performance and project activity.

If you overestimate the tasks, it may lead to the appearance of new risks that can ruin your project plans.

It is worth mentioning that only accepted user stories should be taken into account while evaluating an under/overestimation ratio.

In case you experience underestimation, then you just cannot correctly evaluate how many hours or story points are needed to complete a specific user story and do not meet clients’ needs.

For example, you have estimated 12 hours for a certain user story, but the work was done within 24 hours.

This variation of 12-hour overwork leads to extra expenses for clients and high dissatisfaction.

What are the core reasons for underestimation?

• The team is not familiar enough with the product
• The team experience difficulties with the tech stack
• The team has low skills in estimation.

How to solve underestimation issues?

• Take a training course on advanced estimation
• Conduct a sharing session on the product to better discuss all nuances
• Divide project work into small pieces.

What are the core reasons for overestimation?

• Lack of trust and confidence so that the team is afraid to make mistakes.

Effort estimation variance is one of the most effective Agile Scrum metrics.

It is a valuable metric that allows you to track actual efforts made over the estimated hours to improve future performance.

Let’s take a look at an example of how to measure software quality with effort estimation variance.

Here is a table of sprints, estimated hours, and actual efforts in hours that were spent:

As you can see, only the third sprint has the same amount of estimated and actual efforts put into software development. Other sprints were either over- or underestimated.

So, there are two simple formulas that allow you to calculate over/underestimation values:

• Overestimate = (E-A)/E * 100
• Underestimate = (A-E)/E * 100

where A – Actual effort and E – Estimated effort.

This way, we can easily calculate the effort estimate variance for each sprint in a percent:

Based on the data above, you can safely assume, that Sprint 2 was excessively over-estimated (by 69%!) while Sprint 4 was underestimated.

If you see a negative variance, it is a sign that something is wrong.

Your team either doesn’t have enough knowledge to properly evaluate the complexity of the tasks or lacks details or you should check the individual productivity of team members.

Next, you can define the over- and underestimation limits in percent to draw an effort estimation variance chart and bring into sharp the defects of each sprint.

It allows you to analyze the gaps from sprint to sprint and improve your estimation skills based on the analytics data.

Below you can see an example of an effort estimation variance chart:

Effort Estimation Variance | Software Quality Metrics

Scope Change (SC)

Scope change is another valuable and complementary Agile project metric that allows you to analyze root causes of no meeting commitments.

Literally, scope change helps you track the number of stories added/removed within the project during spring or release.

This metric has 2 key indicators ‘descope’ and ‘scope increase’ that help you determine when and why any mistakes or changes were made.

Here are two formulas to calculate scope change indicators:

• Descope: = (D/C) * 100
• Scope increase = (A/C) * 100

where D – removed SP, A- added SP, C – committed SP.

So, let’s take a look at our example input data and estimate the final variance:

As you can see, sprints 2, 4, and 5 were modified by adding or de-scoping story points.

The key reasons for de-scoping stories are:

• The Product Owner doesn’t prepare enough all required aspects for the upcoming sprint so he/she is forced to make changes down the road
• The changes in the priorities of stories.

Based on the above table, we can calculate the scope change variance:

Ultimately, you can build a scope change chart that more vividly demonstrates and allows you to track the variance of changes from sprint to sprint.

Scope Change | Agile Scrum Metrics

Defect Leakage

As your software product grows, the number of bugs reported will naturally increase and influence the final estimation, client happiness, and team satisfaction.

Is there any way to track the bug ratio within the project scope and improve on it?

Sure, take advantage of defect leakage.

So, it is one of the key Agile metrics which allows you to track the summarized number of defects detected when a sprint or release is done.

Primarily, this metric is about the efficiency of quality assurance processes as most bugs become transparent at the stage of testing the delivered software.

At Ascendix, we have established a company RAG (red, amber, green) status.

It allows us to get a relation coefficient/variance to measure and track a project’s activity quickly and easily.

Different projects require different Scrum metrics due to the fact that net indicators cannot and shouldn’t be analyzed identically across multiple projects.

So, defect leakage is a good example of using RAG status to categorize indicators’ values and measure them more efficiently.

Let’s take a look at how we categorized possible variance values based on our RAG status:

Now let’s consider the example input data table where we summarize the detected sprint defects:

In order to calculate the defect leakage value, we need to use the following formula:

Defect leakage = (Post sprint defect count / defects detected during sprint) * 100

This way, we can measure the variance value for each sprint:

Ultimately, you can build a defect leakage chart that vividly demonstrates the volume of defects at every sprint completed:

Defect Leakage | Quality Metrics in Scrum

Good or bad, all products have defects.

However, you can keep a record of defects found and repaired prior to release to maintain a high-quality standard in your software.

Equally important is to make sure that all defects found after a release or sprint are integrated back into the developers and quality assurance processes.

So, try the defect leakage metric for your project and improve the software quality you deliver.

Why Use Scrum Metrics?

Agile productivity metrics provide Product Owners, Scrum Masters, and developers with the visibility they need to assess quality and track productivity throughout the software development lifecycle (SDLC).

Metrics derived from Agile workflows measure agile processes such as velocity and task board activity.

So, Agile project metrics first help evaluate the project development process and indicate positive and negative points that influence the Scrum team, process, and overall performance.

What’s more, when you detect some problems that slow down the development process, Scrum performance metrics will help you determine the root cause so that you can eliminate it quickly.

However, you should clearly realize that NOT all Scrum metrics and reporting can be efficiently implemented for any kind of a project.

As Scott M. Graffius, an author of ‘Agile Scrum: Your Quick Start Guide with Step-by-Step Instructions’, once said:

This means that different metrics work for different projects and there may be no sense in implementing additional Agile KPI metrics if they do not provide value.

Let’s check the key principles you should follow to make metrics effective to evaluate and improve your project management:

A metric should be used by the team.

Metrics should not be implemented just to follow the Agile Scrum methodology. They should be used by the team to learn, improve, and deliver a more valuable product to customers.

A metric should be based on a conversation.

Again, use Agile software development metrics wisely to start a discussion about the process and roadblocks that negatively affect a team’s performance. Metrics are NOT about numbers; they are about changes and analysis.

A metric should answer concrete questions.

All Scrum metrics that you implement in a project should answer specific questions so that you can understand what should be improved.

A metric should be used together with other metrics.

If you use a single metric to evaluate a project, you may bump into a tunnel vision that obfuscates the reality due to the lack of analytics data.

This way, we recommend using several Agile productivity metrics together to get a balanced picture of your project activity.

A metric should be easy to calculate and understand.

If you find it difficult to use a metric, it most likely will not become useful in guiding daily activities, even if it provides great insights into your team’s work.

What is Agile and Scrum?

Agile mindset is totally about flexibility, simplicity, and high speed. Agile stands for complex software development methodologies based on an iterative approach. It allows you to deliver software incrementally instead of all at once. Agile is about any development process that corresponds with the Agile Manifesto principles. This is a collection of software development concepts established by 14 leading tech specialists back in 2001.

The word ‘agile’ has the following meanings in the Oxford English Dictionary (OED):

• able to move quickly and easily
• able to think quickly and in an intelligent way.

Simply speaking, these are the cornerstones of your business success. If you are not flexible to obstacles and do not make decisions quickly – you are most likely on the competitors’ heels.

To be short, Agile Manifesto values are as follows:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan.

Key Agile Manifesto Principles

What Are The Benefits of Agile Software Development?

Prime quality product

Agile software development with Scrum is divided into multiple sprints during which clients/product owners can make changes to the requirements and functionality.

This way, the chances to get a software product fully compliant with clients’ needs are much higher than following traditional methodologies when on-going changes are impossible.

High customer satisfaction

As the clients are constantly involved in the decision-making and development processes throughout the entire project delivery, they become fully satisfied with the process, result, and reduced time-to-market period.

The latter becomes possible through high involvement from clients and close interaction during the development process.

Controllability

Agile project management provides high transparency, feedback integration, and daily progress reports that help managers and stakeholders better control the project development process.

Better project predictions

It becomes easier to indicate and predict risks when the project development is divided into multiple sprints with continuous improvements and alterations. This way, managers can easily predict sprint-to-sprint changes and plan efficiently.

Low risks

Advanced risk management is a hallmark of Agile software development with Scrum as small sprints and continuous delivery allow to better predict the upcoming development phases and reduce potential risks.

Continuous improvement

As one of the 12 key concepts of the Agile Manifesto, continuous improvement means that managers and developers perform error correction sessions from sprint to sprint and enhance the delivery quality and results of the Agile software development project.

What is Scrum?

Scrum is an Agile framework for developing, delivering, and sustaining complex products, with an initial emphasis on software development, although it has been used in other fields including research, sales, marketing and advanced technologies (Wikipedia).

Put it simply, the Agile Scrum framework is a lightweight methodology that helps a cross-functional and self-managing team to deliver value. It’s also scalable and can be used by 2 or more Scrum teams using Scaled Agile frameworks for more complex projects.

Agile Scrum roles include:

• A Product Owner

An apologist who perfectly knows the Product Goal. Being a part of a Scrum team, the Product owner keeps the needs of many stakeholders in the Product Backlog, which should represent the Product Goal.

So, the main accountability of the Product Owner is to keep Product Backlog items clear, highly transparent, and well communicated.

• A Scrum Master

A person who helps to establish and maintain a Scrum framework for the team.

This person helps Developers accomplish and deliver sprint increments without disturbances and distractions.

Scrum Master’s main accountability is to keep the Scrum team effective.

This is achieved by coaching the team to be self-managed and cross-functional, focusing on goals, resolving roadblocks, and maintaining Scrum events.

• Developers

A self-managing and cross-functional team of specialists that have the required skillset to deliver value each sprint.

In the context of software development, Developers may carry out such activities as prototyping, UI/UX design, software development, testing, etc.

They approach user stories and always follow their priority to deliver the required functionality first.

Agile Scrum Roles | Ascendix Tech

Apart from roles, here are also 4 Scrum events:

• Sprint Planning

The first meeting includes all Scrum team members (Scrum Master, Product Owner, and Developers) to collaborate, discuss the work required to achieve Sprint Goal.

Most often, a Scrum Master facilitates this meeting, where the Product Owner and developers ask/answer questions about execution and acceptance criteria.

• Daily Scrum

A day-to-day meeting during a sprint often takes around 15 minutes. It is organized to schedule the day’s work program towards the Sprint Goal.

Also, Daily Scrum allows developers to discuss current user stories in order to inspect and adapt their working processes on a day-to-day basis.

• Sprint Review

A demonstration meeting at the end of a sprint to show off the product developed during the last sprint to customers and stakeholders.

Simply put, sprint reviews allow stakeholders to ask clarifying questions to better understand what was done in detail and adjust the Product Backlog to better suit the Product Goal.

• Sprint Retrospective

A final meeting before a new sprint where a Scrum team discovers the process, what went right, and how to improve communication, tools, and the process itself to increase the delivery quality and effectiveness.

4 Scrum Events

Key Agile Scrum Artifacts

The framework includes three main artifacts that refer to the kind of work that should be done to finish sprints and the project. They help make internal information transparent for all team members.

Before we start reviewing them, we should mention that each artifact includes a commitment.

According to The Scrum Guide:

So, Agile Scrum artifacts include:

• Product Backlog

The list of required actions that should be completed during the entire project development process to deliver a high-quality product.

The commitment of Product Backlog is a Product Goal.

• Sprint Backlog

All the tasks created for a particular sprint should be done within one iteration. A Sprint Backlog is created based on Product Backlog during a Sprint Planning meeting.

The commitment of Product Backlog is a Sprint Goal.

• Increment

A sum of all Product Backlog elements that were completed during a sprint and the value of all previous sprints. By the end of a sprint, an Increment should be “Done” which means that it is functioning and meets the acceptance criteria of a Scrum team.

The commitment of Product Backlog is a Definition of Done.

Main Agile Scrum Artifacts

Wrapping Up

With the help of the Ascendix Tech Project Management team, we shared with you the Scrum metrics we use when developing software. They can help you better analyze and measure your team’s performance and project progress to improve on.

So, try these Agile metrics for your projects and find those that help you enhance the final software quality and value delivered.

What ways do you use to measure software quality? Do you use any metrics? Share your experience in the comments below.

If you are looking for a reliable software development company with expertise and experience in both creating top technology solutions and effectively managing projects, feel free to contact us.

We will be glad to discuss your project, capture your goals, and transform your idea into a successful custom software on time and on budget.