Part II · Chapter 3 · Risk, without the score · lesson 7

The residual position nobody went back and changed

13 min

Builds on Four things you can do, and only one of them is work

One row, fourteen months old, from a register that has two sets of bands on every line: where the risk sat before anybody did anything, and where it sits now.

A support engineer can read any customer's records without that customer having asked for anything.

Before: several times a year, and a customer leaves. After treatment: several times a year, and a customer leaves.

The control exists. It was built in March, it has been running since, and it has a number against it — every support read now requires an open ticket naming the customer, and the log shows 2,140 reads in eleven months, of which eleven were flagged as having no matching ticket and two were investigated.

The two sets of bands are identical.

Predict

Identical before and after, with a control that demonstrably runs. What does the row tell you?

The control does nothing. Eleven months, no movement.Nobody went back and re-scored it after the control shipped.Nothing. The two explanations produce identical rows.The control works and the bands are too coarse to show it.

A before-and-after pair is a measurement claim. It says somebody assessed the risk, something changed, and somebody assessed it again against the same definitions.

Three things have to have happened for the pair to mean that: the first assessment predates the control, the second postdates it, and both used the same bands. Registers rarely record any of the three.

What they record instead is two columns, both filled in on the same afternoon, often by the same person, often from the same conversation. The pair then reads as a measurement and is a pair of opinions formed four minutes apart.

The tell is cheap. Ask what date each half was written. If it is the same date, the pair is not a before and after at all.

What you would actually do about it

What would you do

You inherited this register last month. The row above is one of nineteen with identical pairs. An internal review is next quarter, you have the ticket-gate logs, and you have about three days you could spend on this before it starts costing somebody else's week.

Optimising for: End able to say something about this control's effect that you would defend out loud, without inventing a number.
You cannot know: What the before bands were based on, since nobody recorded it
Whether the eleven flagged reads would have been visible before
Whether reads happened through other paths the gate does not cover
Whether the two investigated cases were the only real ones
How the reviewer reads nineteen identical pairs

Re-score all nineteen now, dropping the obvious ones two bands.Leave both halves and add a line saying the residual was never revisited.Go to the gate's own logs and ask what the control caught.Drop the residual columns from the register entirely.

The condition under which the pair was never a pair

There is a version of this that is worse than nobody re-scoring, and it is common enough to check for first.

The register was built after the control was already running. Somebody sat down in year two, wrote the risks the organisation has, banded each one as it stands today, and then filled in the before column by imagining the world without the control that was already there.

In that register the before column is a counterfactual invented later, and the after column is the only real assessment on the page. The pair is not a measurement, not a before and after, and not two opinions four minutes apart — it is one opinion and one reconstruction, and the arithmetic anybody does on the two of them describes nothing that ever happened.

The way to find out is to ask when the control shipped and when the register was started. If the register is younger, every pair in it is this.

What this workspace does instead

The register in this subject holds one position per risk. There is no residual pair and nowhere to put one, which is a real limitation and worth saying plainly: you will meet the two-column format everywhere else, and when you do, the questions above are the ones to ask of it.

What you can do here is keep the sentence current. When a control ships, the honest edit is often to the risk itself rather than to a band — the path changed, so the sentence changed, and a new sentence with today's date is more information than a lower number with no date at all.

Worth remembering

A before-and-after pair is a measurement claim, not a formatting habit.
Both halves written on the same date are two opinions, not a measurement.
A register cannot distinguish a failed control from an unrevisited score.
A control that cut the rate by two thirds may not cross a band.
The useful information lives in the control's own logged instances.
A register younger than its controls has an invented before column.
When a control ships, update the sentence, not only the band.