This post was originally an email sent to the Better Questions Email List. For more like it, please sign up – it’s free.
——
How worried should I be that I have 8 months to live?
——
So far, we’ve discussed how we make decisions involving probability:
– We form beliefs about how likely certain events are based on our own experiences.
– These beliefs are “Priors.”
– They act as a map, guiding us through uncertain territory.
– The base rate is the probability of an event occurring. To find it, we look at the average outcome of similar events in the past.
– Base rates are often far more useful for decision making than our priors.
– But because people love narratives – individuating data – we often ignore base rates. We say “it’s different this time.”
So:
Is decision-making just a matter of asking “what’s the base rate?”
No.
Because just as our priors can be misleading if they are not representative…
Base rates can be misleading as well.
Imagine this scenario:
You’re a 20 year old personal trainer. Let’s call you Jill.
(You remember Jill – our personal trainer slash administrative assistant from last week?)
This is your fifth trip to the doctor’s office this month.
You wait, nervously sipping a protein shake.
It’s been days of invasive tests, blood samples, medical forms. Will they finally be able to find what’s wrong with me?
The door opens. Your doctor enters.
You look up hopefully, but her face is grim. Stoic. Professional.
She sights, sits down in front of you, and says:
“I’m sorry…but based on your test results, you most likely have Barrett’s Syndrome.
It’s a rare condition characterized by having a gigantic brain and devastatingly-high levels of attractiveness.
There is no known cure.”
You lower your protein shake. Is the room spinning? Your skin feels flush.
“What’s the prognosis, doc?”
She looks you right in the eyes.
She’s both empathetic and strong. Wow, she’s good at this, you think.
“The average life expectancy of someone with Barrett’s Syndrome is…
….8 months.”
The room goes dark. She has a kind voice.
Your last conscious perception is of your protein shake, falling to the floor and spilling everywhere.
Let’s ask an important question:
How worried should Jill be?
On the face of things, she should be pretty worried.
The base rate here is clear:
The average patient lives for 8 months after receiving this diagnosis.
But all averages are not created equal.
To help us understand Jill’s predicament, we need to bring in two important mental models:
Mean vs. median…
And Individual indexing vs. Group indexing.
Let’s start with mean and median.
There are different measures of “average,” or central tendency:
The median, which is how most of us think of averages, is the total number of items divided by the number of sharers.
The mean, another way of measuring “central tendency,” is the middle point.
If you line up 5 kids by height, the middle child will be shorter than two and taller than two. That’s the mean.
When we hear “average life expectancy of 8 months,” our natural reaction is to extrapolate.
We think “the average is 8 months, so I am likely to have only 8 months to live.”
But what if that 8 months is the mean?
That would mean that half of people would live longer than 8 months. It all depends on which average we’re talking about.
Does this sound far-fetched?
I’d agree with you, except this exact thing happened to evolutionary biologist Stephen Jay Gould.
From Gould’s wonderful essay, The Median Isn’t The Message:
“In July 1982, I learned that I was suffering from abdominal mesothelioma, a rare and serious cancer usually associated with exposure to asbestos.”
The literature couldn’t have been more brutally clear: mesothelioma is incurable, with a median mortality of only eight months after discovery. I sat stunned for about fifteen minutes, then smiled and said to myself: so that’s why they didn’t give me anything to read. Then my mind started to work again, thank goodness.
When I learned about the eight-month median, my first intellectual reaction was: fine, half the people will live longer; now what are my chances of being in that half. I read for a furious and nervous hour and concluded, with relief: damned good.”
We tend to think of averages as “real” – as concrete things, out there in the universe.
But they aren’t.
Averages are abstractions – a way of thinking about the world. They aren’t determinants.
And while the average doesn’t truly exist except in our minds…
Variations around the average are all that exist.
It reminds me of these images of “the average” person that made their way around the internet a few years ago:
This person is a figment; they don’t exist. And we know that.
But it’s hard to shake the feeling that that “8 months” number means something for Jill.
After all, it isn’t based on nothing. So how do we use it?
Let’s come back to Gould. He’s been diagnosed with mesothelioma, which has a mean survival time of 8 months.
If the mean is 8 months, then half of mesothelioma patients must live longer than that.
But which half?
“I possessed every one of the characteristics conferring a probability of longer life: I was young; my disease had been recognized in a relatively early stage; I would receive the nation’s best medical treatment; I had the world to live for; I knew how to read the data properly and not despair.”
The 8 months survival number is a group index.
It accounts for everyone and mashes their survival rates together.
But Jill isn’t everyone; she’s not the average.
What we need to know is:
What’s the survival rate for people like Jill?
That’s an individual index.
Jill’s healthy, she’s young, she’s fit.
And because of this, Jill is likely on the other side of the mean.
Back to Gould:
“I immediately recognized that the distribution of variation about the eight-month median would almost surely be what statisticians call “right skewed.” (In a symmetrical distribution, the profile of variation to the left of the central tendency is a mirror image of variation to the right.
In skewed distributions, variation to one side of the central tendency is more stretched out – left skewed if extended to the left, right skewed if stretched out to the right.) The distribution of variation had to be right skewed, I reasoned.
After all, the left of the distribution contains an irrevocable lower boundary of zero (since mesothelioma can only be identified at death or before). Thus, there isn’t much room for the distribution’s lower (or left) half – it must be scrunched up between zero and eight months. But the upper (or right) half can extend out for years and years, even if nobody ultimately survives.”
In fact, Gould ended up living for another twenty years – before eventually succumbing to a different disease.
While averages are useful, we always need to account for the ways in which our situation is not average…
In other words, we need to take into account our individuating data.
Here is where we try to bridge the gap between map and territory:
By understanding that our priors can be useful…
But only when used to further our understanding of the base rate.
Combining the narrative power of human thought…
With our ability to see patterns at a high level…
…is how good decisions are made.
More art than science.
1
Leave a Reply