Scales built by Experts and Decision-Making
In Chapter 21 of Thinking Fast and Slow (2011) “Expert Intuitions vs Algorithms”, Daniel Kahneman summarizes research and insights kicked off by psychologist Paul Meehl in the 1950s.
I had a personal experience with the Chapter 21 ideas this past week, making a decision under stress.
“Several studies have shown that human decision makers are inferior to a prediction formula even when they are given the score suggested by the formula! They feel that they can overrule the formula because they have additional information about the case, but they are wrong more often than not. According to Meehl, there are few circumstances under which it is a good idea to substitute judgment for a formula.”
(Kahneman, Daniel. Thinking, Fast and Slow (p. 224). Macmillan. Kindle Edition.)
Kahneman then explicates the remarkable contribution by Robin Dawes who showed that scores derived from an equal weighting of factors outperformed regression-based formulas in many prediction applications. (“The Robust Beauty of Improper Linear Models in Decision Making” American Psychologist, July 1979, Vol. 34, No. 7,571-582, http://www.niaoren.info/pdf/Beauty/9.pdf )
“The surprising success of equal-weighting schemes has an important practical implication: it is possible to develop useful algorithms without any prior statistical research. Simple equally weighted formulas based on existing statistics or on common sense are often very good predictors of significant outcomes.”
(Kahneman, Daniel. Thinking, Fast and Slow (p. 226). Macmillan. Kindle Edition.)
Kahneman illustrates this point with the story of Virginia Apgar, who used her expert knowledge to draft and refine the Apgar score that enables a physician or nurse to rate the health of newborn babies. Low scores call for intervention by the care team.
An algorithm for decision-making is even more useful when the decision-maker faces psychological challenges. Stress, for example, appears to reduce at least some aspects of cognitive function. (See for example C.S. Mackenzie et al. (2007), “Cognitive Functioning Under Stress: Evidence from Informal Caregivers of Palliative Patients”, Journal of Palliative Medicine, 10(3): 749–758. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1905830/ )
Andy’s Story
My family and I benefited from the power of a simple equal-weighted scoring framework last week, as we cared for our old dog Andy. His picture sits at the top of this post.
My post in January described Andy’s diagnosis with lymphoma. The intervention proposed by our vet gained Andy a few weeks of relief; during that time, Andy had more good days than bad, with healthy appetite and interest in going for slow walks, happy to see people and other dogs.
Last week, we learned that Andy’s lymphoma was progressing. He got weaker and his kidneys were not functioning normally. He had trouble getting up without help, needed to rest frequently on a short walk, and was losing interest in eating his favorite foods. We had no prospect of doing anything to halt his continued decline, much less reverse it.
Andy’s personality and intelligence made him a great companion over our 13 years together; we knew we had to interpret his symptoms and behavior to make decisions about continuing or ending his life.
On Thursday morning, I used an eight-factor quality of life scoring system developed by Katie Hilst, a veterinarian in Madison, Wisconsin who specializes in end-of-life care for domestic animals (www.journeyspet.com). Dr. Hilst built her scale from extensive conversations with pet owners and families.
The low score I calculated helped frame my observations and intuition as we thought through how best to care for Andy in the face of uncertainty, sadness and stress.
We made the last call to our vet late on Thursday, who came to our home to euthanize Andy that evening. Dr. Hilst’s explicit decision framework provided an anchor in an emotional swirl, a rational complement to our intuition.
The insights of Meehl and Dawes played out in a real way last week with Andy and convince me of their relevance and power.
