Measuring Quantity of Work

ABOUT THIS CHAPTER   Less effective Agile implementations sometimes treat measurement as the enemy. More effective Agile implementations use measurement to include quantitative data in their process change decisions, rather than basing decisions on subjective opinion alone.

This is the first of three chapters that discuss quantitative approaches to Agile development. This chapter describes how to establish a meaningful measurement baseline. “More Effective Agile Process Improvement” discusses how to use the measures for process and productivity improvement. “More Effective Agile Predictability” discusses estimation.

Measurement begins by measuring how much work is being done. On Agile projects, that means measuring work-item sizes in story points. A story point is a measure of a work item’s size and complexity. Agile teams use story points primarily for estimating, planning, and tracking their work. Story points are useful for measuring process improvements and productivity improvements, too.

Agile teams most often use story point scales based on the Fibonacci numbers from 1–13 (1, 2, 3, 5, 8, and 13). Each work item is assigned a size in story points, and the sizes of the individual work items are added to arrive at the work’s total size in story points.

Non-Fibonacci values, such as 4 and 6, are not used. This helps to avoid the false precision of debating whether a story is a 5 or a 6 when the team doesn’t really even know whether the story is a 3, 5, or 8.

In an ideal world, there would be a universal standard for how each story point is measured and assigned. But in the real world, each team defines its own scale for how large a story point is on that particular team. After working with its scale a while, the team gets synched up on how big a 1 is, how big a 5 is, and so on. Most teams need experience actually assigning story points before their story point scale stabilizes.

Once story points have been assigned, the team does not change story point assignments based on actual performance. If a story was assigned 5 story points initially but feels more like an 8 by the time it’s completed, it remains a 5.

Velocity

Once the size of the work has been established through story points, the next step is to calculate the rate at which work is completed.

On Agile teams, “story points per sprint” make up the team’s velocity. A team that completes 42 story points in a sprint has a velocity of 42 for that sprint. A team that completes 42 story points in one sprint, 54 in the next sprint, 51 in the next, and 53 in the sprint after that has an average velocity of 50.

Velocities of individual sprints fluctuate and are not usually meaningful. Trends in average velocity over time are more revealing. Once the team has established a baseline velocity that it believes accurately represents its work-completion rate, the team can begin experimenting with process changes and observe how those changes affect its velocity. How to do that is discussed more in the “More Effective Agile Process Improvement” chapter.

Some teams also track scope velocity, which is the rate at which work is added to an ongoing project.

Small stories

For general use (not in support of measurement), some teams will use additional story point values—such as 21, 40 and 100 (round numbers) or 21, 34, 55, 89 (Fibonacci numbers)—to represent themes, epics, and larger backlog items.

For the sake of supporting meaningful measurement, stories should be decomposed so that they fit onto the scale of 1–13, and teams should take care to apply the story points proportionately. A story that’s assigned 5 story points should be about 5/3 as large as a story that’s assigned 3 story points. This allows a team to perform meaningful numeric operations, such as adding up total story points.

Numbers like 21, 40 and 100 are not meant to be used in the same way. They are more metaphorical than numeric, and for measurement purposes they should be avoided.

Short iterations

Velocity is computed on a per-sprint basis, so the shorter you make your sprints, the more often you can update your team’s velocity. When compared to Sequential software development, in which whole-lifecycle iterations can take quarters or years, and therefore require quarters or years to fully calibrate a team’s productivity, short iterations allow a team’s velocity to be calibrated within only a couple of months.

Comparing teams’ velocities

Each team creates its own story point scale, based on the specific kind of work it’s doing. Leaders naturally want to compare team performance, but there are too many differences in teams’ work to allow for meaningful cross-team comparisons. Teams vary based on:

• Different kinds of work (greenfield vs. legacy, front end vs. back end, scientific vs. business systems, and so on)

• Different technical stacks, or different parts of the same stack

• Different stakeholders who provide different levels of support

• Different numbers of team members, including team members being added and subtracted at different times

• Different responsibility for production support

• Different exceptions to their usual velocities because of training, vacation schedules, release schedules, holiday schedules at different geos, and other factors

Even though all teams are using story points, it is not meaningful to compare one team’s velocity to another. It’s as though one team is playing baseball, another team is playing soccer, and another is playing basketball. Or one is playing NBA basketball, and another is playing summer league basketball. Comparing runs, goals, and points across teams is not meaningful.

Leaders who have tried to compare team performance by using velocity say that the effort is toxic. It pits teams against one another. Teams are aware that the comparisons are based on unreliable data, so they regard the comparisons as unfair. The result is a reduction in morale and productivity—the opposite of the purpose of comparing the teams in the first place.