Track 11 · Noise · lesson 1
A good decision can have a bad outcome
11 min
Track 11 · Noise · lesson 1
11 min
Someone drives home after a long evening, decides they are fine, and arrives without incident. Nothing happened. No police, no accident, no near miss.
Ask whether it was a good decision and almost everybody says no, immediately, without hesitating. Now change one detail: the same person makes the same decision and puts the car into a wall.
Nothing about the decision changed. Everything about how it gets judged did.
A decision is made with the information available before the outcome exists. An outcome is produced by the decision and by everything else — other people, the weather, timing, and a great deal of chance.
Judging a decision by its outcome is judging it by inputs the decider never had. It also teaches the wrong lesson in both directions: it rewards reckless choices that happened to land and punishes careful ones that happened not to.
Every decision, once the outcome is known, sits in one of four places.
Good decision, good outcome. The one everybody wants, and the one that teaches least, because it is indistinguishable from luck at the time.
Good decision, bad outcome. The most under-appreciated box. The correct call at the correct odds still loses a large share of the time, and if you review it as an error you will stop making it.
Bad decision, good outcome. The dangerous box. Something reckless was rewarded, and the reward is a training signal pointing directly at doing it again, harder.
Bad decision, bad outcome. The easy one. The lesson arrives with the consequences attached, which is why this is the only box that reliably produces learning.
Predict
The outcome is loud and the process is quiet.
An outcome is a fact, it has a number, it arrives on a specific day and other people can see it. A decision process is invisible, has no natural units, and is only assessable by reconstructing what was known at the time — which is exactly the thing that becomes impossible once you know how it turned out. The next lesson but one is about why.
So the outcome is the only variable that is easy to observe, and it gets used as the measure. Bonuses are paid on it, reputations are built on it, and sensible organisations full of intelligent people do this constantly.
Not the same thing
Whether the choice was sound given the information and options available at the time.
Whether the result was good.
Which is which? Put each one on a side.
A business that failed after the founder checked demand, priced honestly and kept costs low
A single holding that tripled in a year
A written record of what someone expected before they acted, checked afterwards
The advice "judge the process, not the outcome" is correct and almost useless as stated, because the process is not observable from where you are standing. Three things are genuinely available.
Ask what else could have happened. Not as a consolation, as a description. If a decision had one plausible path to a good result and five to a bad one, landing on the good one does not make it a good decision — it makes it a lucky one, and the ratio is the thing to notice.
Look at the size before the direction. Sizing is the part of a decision that is almost entirely under the decider's control, and it is therefore the part where the outcome tells you the most. Someone who bet an amount that would have ended them was making a bad decision whether or not it landed.
Use samples, not instances. One outcome is noise. Twenty outcomes from similar decisions start to say something, which is why calibration needs a track record and why one spectacular result says almost nothing about whoever produced it.
Check
Deciding well and finding out are separated by different amounts of time in different fields. A chess move gets assessed in minutes. A hiring decision takes a year. A decision about where to put money for a working lifetime takes thirty.
Where the gap is short, outcomes are a workable proxy for decisions, because the noise averages out over many observations. Where the gap is long, a person may make fewer than ten decisions of a given type in their whole life, and ten observations from a noisy process is not a sample — it is anecdotes.
That is the structural reason financial judgement is hard to acquire by experience alone, and the reason a written record beats a memory.