Part III · Chapter 5 · Evidence · lesson 12
Defend one artefact against the obvious question
15 min
Builds on Two different things evidence can show, and only one is hard
Part III · Chapter 5 · Evidence · lesson 12
15 min
Builds on Two different things evidence can show, and only one is hard
Pick the weakest piece of evidence you hold. Not the weakest control — the weakest artefact. You already know which one it is; everybody does.
Somebody has just looked at it and said, politely, that they are not sure it establishes what you say it establishes.
The reflex is to argue that it does. The move that works is different, and it has four parts.
Predict
Four sentences, in this order, and the order is doing work.
What it shows. The narrowest true claim. A state, or an event, with a date and a scope.
What it leaves open. The questions the artefact cannot answer. Duration, completeness, whether the scope shown is the whole scope.
Who produced it, and could they have changed it. The dating question from earlier in the track, answered before anybody has to ask.
What you would offer instead. The next artefact up, or the honest statement that there is not one and what would have to start happening for there to be.
What this cannot do is tell you whether it will do. Nobody here can know that — not the course, not the register, not you. What the four sentences achieve is that the person deciding is deciding about the artefact rather than about how carefully you described it.
The control below is the hardest one in the registry to evidence, which is why it is the one to defend. It runs continuously, so there are no instances to count; the design evidence is excellent and cheap; and the operation evidence is a question about whether a human was awake.
Where they part company
The alert someone actually answers
Detection and the evaluation of what was detected are one control on the management-system side and two criteria on the other, so the same alert history gets tested once there and twice here. The consequence is asymmetric: a gap in triage records is survivable where monitoring is a technological control and shows up from two directions where response is separately criterion-bound.
Decide · monitoring-that-reaches-someone
A detection that fires into a channel nobody reads has the same outcome as no detection, and costs more. This control is the path from a signal to a human who is awake, authorised and expected to do something, and the measure of it is how long that path takes in practice rather than whether it is drawn.
The decision
What is worth waking somebody for. Every alert added to the paging route makes the route slightly less trusted, so the real decision is which signals get demoted to a queue and who reviews the queue. An organisation with no out-of-hours cover has to decide that deliberately and say what the consequence is, rather than leave the rota blank.
Where this goes wrong
Alert volume grows faster than the attention available to it, and the adaptation is individual and invisible: people stop looking rather than the rota changing. Nothing in the configuration records the change, so the control's design evidence stays accurate while its operation quietly stops, which is the hardest version of this failure to catch from the inside.
Does this apply to you?
What does not line up
Nothing yet, because there is nothing in here yet. An empty workspace has no contradictions and tells you nothing.
These are contradictions between things you wrote. Nothing here can tell you whether any of it is enough — that is a judgement, and it moves with the auditor, the boundary and the year.
Take the usual bundle for this control and run it through.
What it shows. The paging configuration, exported on a date: which conditions route to a person, which go to a queue, and who is on the rota. That is a real and complete claim about design, and it is as strong as design evidence gets.
What it leaves open. Everything about operation. Whether any alert fired. Whether anybody acknowledged one. How long it took. What happened to the ones nobody acknowledged, which is the interesting population and the one that is never in the bundle. A configured route and an answered page are different events and only the first is in the export.
Who produced it. An export from the tool, dated by the tool, which is better than most things in the folder. Worth saying so, because this is the one of the four where this artefact is strong.
What you would offer instead. The alert history with acknowledgement times and names, including the unacknowledged ones. If that exists, it is the whole answer and the conversation ends. If the retention on it is thirty days and your period is nine months, say that — and notice that the sentence you have just said is the real finding, arrived at in four lines rather than four weeks.
There is one version of this where the four sentences run out, and it is worth rehearsing because it is common and it is survivable.
The question is about a period before you started keeping the artefact at all. The paging tool was adopted in March; the period opens in January. There is no artefact for January and February, there is no artefact that could be produced for January and February, and no amount of describing what you hold changes that.
The honest answer has three parts and none of them is an argument. What existed before: probably a channel somebody watched, with nothing exportable. What changed and when: the adoption date, which is a fact with its own artefact. What is true from that date forward: the records exist, here they are.
What you must not do is let the shape of the bundle imply coverage the dates do not support. A folder of alert histories starting in March, filed under a period starting in January, makes a claim nobody wrote down and everybody can check.
Check
The strongest artefact in your bundle teaches nothing. It shows what it shows, nobody questions it, and the four sentences collapse into one.
The weakest one is where every distinction in this track is live at once. It is usually design evidence being asked to carry an operation claim. It usually has a date somebody in the organisation could have set. Its scope is usually narrower than the control it is filed under. And the question of what to offer instead usually has no answer, which is how you find out that the gap is in the control rather than in the paperwork.
Which is the real output of this lesson, and it is not a better sentence about an artefact. It is a short, specific list: the two or three controls where nothing stronger exists and nothing stronger can be produced without changing how the work happens. That list is what the next year of this is actually for.