Part II · Chapter 4 · What a control is · lesson 1

The document is the intention; the control is the thing that happens

12 min

Somebody sends you a twelve-page document. It is called an access policy, it has a version number, an approval date in February and the CTO's name at the bottom. It says that privileged access is reviewed every quarter.

Read it twice. Then answer one question about the year it covers: how many reviews happened?

Predict

The policy says quarterly. It was approved in February. What does the document tell you about how many reviews actually took place?

Four, give or take — the policy was approved and policies get followedThree, because the first quarter was already partly goneNothing at allAt least one, since the policy would have been tested before approval

Two different questions get asked about every control, and almost all the confusion in this field comes from answering the first and believing you answered the second.

Design: is there a thing here, aimed at something, with somebody meant to do it, at a stated frequency, in a way that would address the risk if it happened? A policy, a diagram, a configuration export and an approval all speak to this.

Operation: did it happen, during the period in question, often enough, and is there an artefact that would show it to a stranger? Only something produced by the activity itself speaks to this — a dated record, a completed review with a name on it, a log the work emitted as a by-product.

A document can be perfect design evidence and zero operation evidence at the same time. That is not a flaw in the document.

The asymmetry runs one way

Design without operation is extremely common and looks exactly like progress. Somebody writes the policy, circulates it, gets it approved, files it, and the register gains a row saying the control is in place. Nothing is dishonest anywhere in that sequence.

Operation without design is rarer and reads strangely: the activity happens every month, reliably, because one person has always done it, and no document anywhere describes what they do or what would happen if they stopped. That organisation has the harder half and will be asked for the easier one.

The two halves also fail differently. Design decays when the organisation changes shape and nobody reopens the document. Operation decays when somebody gets busy, and it does so without a single visible event.

Something people say

“We have the policy, so we have the control.”

Why it spreads: Because the policy is real, and it is the only artefact most organisations can produce on demand. It has a date, an approver and a version, which are exactly the properties that make something look like a record. It is also the part of the work one person can finish in an afternoon, so it is the part that gets finished.
The part that is true: The policy genuinely is evidence, and a register is genuinely better for having it. It establishes that a decision was made, what the decision was, who agreed to it and when. Without it, the activity — even performed faithfully every month — has nothing stating what it is for or what cadence it owes. Half of what gets asked about a control is answered by the document and only by the document.

What exactly is missing when a policy is the only artefact attached to a control?

Not the same thing

Evidence of design

An artefact showing that a decision was made about how something would be done.

Measured in: Whether it describes the control: scope, owner, cadence, and what happens on an exception.
For example: The written review cycle, naming which systems are in scope and who signs each one off.
Chase it alone and: Accumulates. An organisation can hold thirty documents and have no idea which of the activities they describe are still happening.

Evidence of operation

An artefact the activity itself produced, on a date, that nobody had to write up afterwards.

Measured in: Whether a stranger could count instances from it and place each one inside a period.
For example: A completed review for a named system, dated, with the reviewer's name and the entries they removed.
Chase it alone and: Exists only if the control ran, so it is absent exactly when it is most wanted and cannot be produced retrospectively without lying.

Which is which? Put each one on a side.

A diagram of the deployment pipeline, current as of last month
The export of who holds standing administrative rights today
A restore test from August with the elapsed time and the problems found
The approved incident threshold, with examples on both sides of the line
Alert acknowledgements over six months, including the ones nobody acknowledged
A dated acknowledgement from each member of staff of the current terms of use

Two of those are the ones people argue about, and the argument is worth having out loud. A configuration export feels like hard evidence because it came out of a machine. What it shows is a state at the moment of export, and a state is the result of a design decision rather than a record of an activity. It leaves open the question of what the setting was in March.

Why this gets a track rather than a paragraph

The distinction does not survive being explained. A reader who has nodded at the paragraph will still, two weeks later, attach a policy to a control marked as running — not out of carelessness, but because the policy is the artefact in front of them and the gap is invisible from where they are standing.

What does seem to survive is having gone through your own controls, one at a time, and found the ones where the honest answer to the question of what you would hand over is a document. That is the next lesson, and it is uncomfortable in a useful way.

Check

A control is marked as running. The only artefact attached is a policy, approved last year, describing a monthly activity. What does the register currently show?

That the control is designed, and that the claim it operates has nothing attached behind itThat the control is not operatingTwelve instances, implied by the cadence in the policyNothing, since a policy is not evidence

The one case where operation evidence is genuinely hard to defineOptional

A control that runs continuously has no instances. Nobody performs it on a Tuesday; it is a condition that either holds or does not.

Branch protection is the cleanest example. There is no monthly instance of branch protection. What exists is a setting, and a population of changes that went through while the setting was in force.

For controls like that, operation evidence gets built sideways: the setting's own change history, showing it was not altered during the period, plus a sample from the population it governed. Both are produced by something other than the control itself, which is why these are the controls where a register's language gets loosest — and why the lesson after this one names that condition explicitly.

Worth remembering

Design evidence shows a decision was made about how something would be done; operation evidence shows the thing happened on dates inside a period.
A policy is real evidence of design and no evidence of operation, and both halves of that sentence matter.
Only something the activity produced as a by-product speaks to operation — a record written up afterwards is weaker because it did not have to exist.
A configuration export shows a state at the moment of export, which is design evidence wearing the costume of a record.
Design decays when the organisation changes shape; operation decays when somebody gets busy, and the second decay has no visible event attached.
A control that runs continuously has no instances to count, so its operation evidence has to be assembled from the setting's change history plus the population it governed.