If Six Measures are Good, Wouldn't 12 or 24 Always Be A lot Better?
Stimulated by a project informed by Brian Maskell’s work on Lean Accounting (introduced here), I’ve been thinking about the number of measures needed to guide managers and improvers.
To manage a value-stream, Brian and his colleagues at BMA, Inc. recommend just six measures to get started:
“It is important to have few measurements. To focus people's attention and motivate continuous Lean improvement, we must select a few well-chosen measures. The measurements also provide a balance of information, are easy to understand and use, and reflect Lean issues.”
(Maskell, Brian H. et al. Practical Lean Accounting, 2nd Edition. Productivity Press, 2011-08-15. VitalBook file, p. 149. For his manufacturing clients, Brian starts with these measures: (1) Sales per person; (2) On-time delivery; (3) Dock-to-dock time; (4) First Time Through aka first pass yield; (5) Average cost per unit; (6) Accounts Receivable days outstanding.)
The API authors of The Improvement Guide (2nd edition) give similar advice for people applying the Model for Improvement:
“Multiple measures are almost always required to balance competing interests and to help ensure that the system as a whole is improved. Try to keep the list to six measures or fewer. Strive to develop a list that is useful and manageable, not perfect.” (p. 95)
Six measures take less work than 12 or 24
Maskell and my API colleagues both know a lot about measurement. They all recognize that measurement typically costs time and money.
First, there are operational costs to acquire data and maintain adequate quality of measurement so this week’s dot on a chart has consistent meaning with the dot from last week. Somebody has to do that work and get paid to do so.
There’s also psychological cost: each additional measure imposes a task burden on decision makers, taxing mental and emotional capacity to integrate additional information, draw useful inferences, and make decisions.
Cost considerations aside, are there any reasons to believe that five or six measures may be enough to guide managers as they work to maintain and improve system performance?
Evidence from the Center for Adaptive Behavior and Cognition: Less sometimes is more
Researchers at the Center for Adaptive Behavior and Cognition summarized several of their early studies in Simple Heuristics That Make Us Smart (Evolution and Cognition), published in 1999.
They describe “frugal and fast” heuristics—rules of thumb, useful shortcuts or approximations—as particular methods to search and generate answers to problems.
In certain situations that demand prediction in an environment with incomplete information, they demonstrate that methods that use a relatively small number of cues or problem features can outperform methods that use many more features.
“A computationally simple strategy that uses only some of the available information can be more robust, making more accurate predictions for new data, than a computationally complex, information-guzzling strategy that overfits.”
“Robustness [making accurate predictions for new data] goes hand in hand with speed, accuracy, and especially information frugality. Fast and frugal heuristics can reduce overfitting by ignoring the noise inherent in many cues and looking instead for the ‘swamping forces’ reflected in the most important cues. Thus, simply using only one or a few of the most useful cues can automatically yield robustness. Furthermore, important cues are likely to remain important. The informative relationships in the environment are likely to hold true when the environment changes…”
(Gerd Gigerenzer;Peter M. Todd;ABC Research Group. Simple Heuristics that Make Us Smart (Evolution and Cognition) (Kindle Locations 336-339). Kindle Edition.)
Near the end of chapter 5, the authors observe: “The fact that a heuristic can disobey the rational maxim of collecting all available information and yet be the most accurate is certainly food for thought.” (Kindle Locations 1512-1513)
Back to Measures
The problem environments described in chapter 5 of Simple Heuristics that Make Us Smart are all decisions comparing two instances. The decision task is to answer the question “Is A better than B?” given multiple cues or factors where there is uncertainty and incomplete information. Furthermore, a few cues are important and capture the main information in the system and are likely to remain important in the near future.
The chapter 5 decision task resembles the situation faced by a value-stream manager or improvement team. The general question is a comparison of the value stream or system with itself (A versus B): is the value stream or system better or worse this week than last? And managers and improvers in any real situation always have some level of uncertainty and incomplete information. If the value stream or system is approximately stable (in a control chart sense), the causal structure will be about the same week to week.
So here's the question: Can we use a small number of measures to see if the current state of our system is better than the state in the recent past?
And prospectively, about the future: Can we use a small number of measures to predict and manage improvement?
Maskell and API answer yes to both questions if the small number of measures balance competing forces (like cost, quality, delivery, and safety).
The ABC Group’s research provides a bit of theory on why those answers makes sense.
Additional Note
The ABC research looks at heuristics in a way that differs from the research program initiated by Daniel Kahneman and Amos Tversky. The Kahneman-Tversky program examines the ways people typically reason and infer, identifying inconsistencies with decision rules derived from logic and probability theory. The inconsistencies are termed "biases.". The ABC group views typical reasoning and inferences by people as functional methods that may perform well in specific environments, where performance is based on success in predicting outcomes, not alignment with abstract rules of reasoning.
