Building Better Metrics


I have seen my share of good and bad metrics. And I continue to get it wrong. Then I learn and try again. Below are some of the guidelines I am using for developing useful metrics.

  1. TIE METRICS TO BUSINESS GOALS – seems obvious but a metric tells you something, and if that something cannot be tied directly to a business goal, why are you using it?
  2. GET ALIGNED ON THE GOALS BEFORE YOU START – it’s amazing how two people can look at the same goal and interpret it in completely different ways. Make the effort.
  3. BALANCED VIEW – weight your metrics so that you avoid over-rotating on any one or small set of measures. A balanced view of metrics provides a holistic picture.
  4. ORTHOGONAL METRICS – rather than use one complicated metric, develop two or three simpler ones that provide a faceted view of what you are trying to measure. The goal here is to avoid metric gaming and having a few metrics that look at a goal from different points of view. The benefit is simplicity without being simplistic. Measuring a complex system requires different views and synthesis to deliver the big picture.
  5. THE NAPKIN TEST – if you can’t write your metric calculation on a napkin and explain it in less than a minute, it is too complicated.
  6. MEASURE OUTCOMES AND MEANS – you could argue that outcomes alone should be measured. Means offers something else – a measure of effectiveness of the tools and practices being used to drive an outcome. If the outcomes are not what you expect, you have metrics to study the means being used, and if the means are being used as intended. If the means are driving improvement, you can correlate outcomes and means until the marginal utility of a mean tapers off on the effectiveness scale.
  7. INSPECT AND ADAPT – as change occurs, be prepared to change your metrics and even your goals – it is not a plan-up-front-and-follow-through-until-finished exercise. Don’t fall in love with the goals or the metrics.
  8. HUMAN METRICS MATTER – focusing only on the quantitative stuff guarantees you will miss useful information about how people feel. You can use surveys and informal conversations to assess the human side. It matters, so do it. Great results at the expense of people is just bad business.
  9. BOIL IT DOWN – less is more. Find the minimum useful set of metrics and work with that. As you review your data sets, you will discover other things you may need to measure, but these should come from shortcomings in the measures you already have – in other words, unanswered questions about business goals. If you add a new metric, can you remove another one?
  10. TRENDS VS. TARGETS – continuous improvement never stops. Metrics should reflect that by measuring trends. Are we getting better, faster, leaner? By how much?
  11. MAKE METRICS EASY TO COLLECT – there’s nothing worse than spending a pile of time collecting and aggregating metrics. Automate as much of it as you can. CPUs are good at this. Use your energy for interpretation and analysis instead.

Have any great ideas or disagree? Please post a comment.

Advertisements

2 thoughts on “Building Better Metrics

  1. When are the “rajile” t-shirts going to be available? Also, kudos for not using the word “organic” anywhere on this site.

  2. T-shirts can be pre-ordered now. They come in three colours. Black, black, or black. Can I put you down for one in each colour, Fred? Thanks for taking the time to post a comment!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s