Measuring Team Productivity – A basic intro

TL;DR;

Managers want to gauge team performance. Classic metrics make it both hard and useless. They should focus on measuring something else instead: success.

Try these measurement:

– Deploys per day
– Lead time
– Defect escape rate

A CEO once asked me: do I have a good team in my hands?

The company seemed to be doing “ok”. Was there an opportunity for improvement? Was the team really the best they could be? What should we focus on?

Typically people are measured by “classic” metrics such as the following ones:

  • Team utilization
  • Story points
  • Stories delivered (or bugs fixed) over time

Unfortunately none of those mean diddly squat when it comes to measuring performance. The reason is that the customer does not care about any of it.

What do we really care about?

What companies care about (or should care about) first and foremost, is the ability to deliver value to the customer, timely and safely. If you deliver the right value, at the right time, in good quantities: business is good.

So in other words what you want to know is: are we making customers happy?

It should be apparent that customers don’t care if your team is in full utilization, how many story points / features you delivered. Here’s why:

  • The team could be 100% busy, but they could be doing the wrong things
  • Story points should only be used to predict productivity, and never to measure it. What’s the difference? It’s subtle, and I won’t go into it at the moment – just trust me on this for now.
  • What good is it to deliver 100 features, if the the customer does not care about those features anymore?

So what does the customer care about? They want to get what they want, when they want it (now).

Measure Success, not performance

The bad news is that, as it turns out, you can’t really measure performance in the classic meaning of the term.

The good news is that there are other, better measurements.

People want to measure performance because they assume that if they can measure performance they can predict success. Let’s skip performance then, and go straight to success.

The conditions for success

For a team to be successful, they need to be able to deliver value to the customer quickly, timely, safely, continuously.

QuicklyOnce you decide to deliver something, you should be able to turn deliver it as quickly as possible.
TimelyYou should be working today on things your customers want today, not the things they wanted 6 months ago.
SafelyDelivering new experiences should never jeopardize existing ones. (Don’t break old things when deploying new ones)
ContinuouslyYou should always be delivering value to your customers

If you can do the above, you are in good shape.

The good news is that you can measure all of the above. These are the metrics you want to start with:

Lead TimeThe amount of time that passes between when you start working on something and when it’s available in production.A low number is better. Few hours tops.
Deployment TimeThe amount of time it takes for code to go from “ready to deploy” to “deployed in production”. Ideally should be measured in minutes. A typical value is < 30 minutes for small to medium systems.
Deploys Per dayHow many time code is deployed in your production environment.Ideally you should have multiple deploys per day per developer (necessary to have low lead time).
Defect Escape RateHow many defects are found in production vs QAYou want the vast majority of your defects to be caught in QA.

Notice that none of the measurements above tell you if your team is busy, or how many stories they delivered.

What it does give you is a clear picture of wether or not your team has any chance of being successful.

Predictive?

As it turns out, the measurements above, not only tell you if you have the conditions necessary to be successful. If you understand each metric and learn what it’s trying to teach you, then hitting the metric will also be a predictor of success. That’s right: hitting the metric and keeping your team on target will likely mean success for you. The reason is that by staying on target with the metric your team will have no choice but do the right thing!

So in case it wasn’t clear, here’s what each metric is trying to teach you:

  • Lead Time should be low because we want everybody to be pushing their code to production as frequently as possible, as quickly as possible. You have a bug for a production issue? Do you want to wait days or hours (or minutes) for you to be able to deploy it to your customers?
  • Deployment time should be low because … well, because Lead Time has to be low, and deployment time is a constant price to pay every time we want to deploy something.
  • Deploys per Day. If your developers don’t deploy value multiple time a day, it means that they are not creating value for your customers quickly or timely.
  • Defect escape rate: by keeping it low you ensure that you are safely delivering new value.

But…

At this point some people ask: how can I have developers deliver to production multiple times a day? Isn’t that risky? It may take days for a developer to create a new experience, or a new feature, or fix a bug. How can he/she deploy anything in the meantime?

It’s a good question, especially if you have been used to a number of anti-patterns. That will be the topic of the next post…

Leave a Comment