Part I · Chapter 1 · The Ask · lesson 2

What the document says, and the much larger thing it does not

11 min

A buyer's vendor-risk team has sent you somebody else's report as an example of what they want. It is sixty pages. Four of them are the part that carries the assertion; the rest is description, tables and a long appendix.

Most people read the four pages as a statement that the company is secure. It is not that. It is narrower, stranger and considerably more useful once you can see the shape of it.

Predict

One of these is close to what the document actually asserts. Which?

An independent view that the company is secureAn independent view on whether the controls the company described were built to do what the description says, and ran that way across a stated stretch of timeA list of the standards the company has metA guarantee that an incident will not happen

Four qualifiers, and each one shrinks the claim.

Described controls. The organisation says what it does. The reviewer forms a view about those statements, not about the controls it never mentioned.

A boundary. The description covers some systems and people and not others. Nothing outside the line was looked at, and the line is in the document.

A period. The view is about a stretch of time with two dates on it. It says nothing about the week after it closed.

Design, then operation. Two separate questions. Whether the control was built to do the job, and whether it then did it. A policy document answers the first and is silent on the second.

Strip all four away and you have a statement about security. Put them back and you have the actual document, which is a much smaller thing to be confident about.

The much larger thing it does not say

Nothing in it speaks to what happens tomorrow. A period that closed in March describes March. If the person who ran a control left in April, the document is not wrong and it is not current either, which is why the gap between the closing date and today is its own conversation later in this subject.

Nothing in it speaks to what sits outside the boundary. A company can draw a narrow line, describe what is inside it accurately, and be entirely silent about the system that actually holds your data — and the document will not look unusual, because the boundary is a paragraph on page three that most readers skip.

And nothing in it ranks you against anybody. There is no grade, no tier and no number. Buyers want one badly enough that several products have been built to invent one, which is the subject of the second half of this track.

Something people say

“They sent us their report, so they are secure.”

Why it spreads: Because it is mostly a reasonable inference, and because the alternative is reading sixty pages about a system you will never see. A company that produced one of these has, almost always, done real work. The inference is not mad — it is only much weaker than the confidence people attach to it.
The part that is true: An organisation that got through this has a boundary written down, controls somebody owns, and artefacts with dates on them. That is genuinely more than most companies have, and it is a real signal about the organisation.

Which part of that sentence does the most unearned work?

Not the same thing

What the document asserts

An independent opinion about described controls, inside a stated boundary, across a stated period.

Measured in: Dates, a boundary statement, and a list of exceptions
For example: Across the twelve months ending in March, the access reviews described in section three were performed, with one exception noted in the second quarter.
Chase it alone and: It is narrow enough that a reader who wanted a verdict will supply one of their own, and the supplied verdict is always more generous than the text.

What the reader hears

A pass mark issued by somebody official, covering the company as a whole, valid now.

Measured in: The presence of the document
For example: They have got one, so that box is ticked and we can move on to pricing.
Chase it alone and: The boundary, the period and the exceptions all vanish, so the one page that would have changed the buying decision is the one page nobody opened.

Which is which? Put each one on a side.

Whether the organisation described its own controls accurately
Whether the company will still be doing these things next month
Whether a control that ran eleven times out of twelve is described as having run eleven times
Whether this company is more trustworthy than a competitor without the document
Whether the systems holding your particular data were inside the line

Check

A vendor's report covers a period that ended five months ago, and the system you are buying is named in the description. What is the honest reading?

It is out of date and therefore worthlessGood evidence about those months, silent about the five since, and the gap is a thing you can ask aboutFine, since these things are valid for a yearEnough on its own, given the system was named

Worth remembering

The assertion is about described controls, inside a stated boundary, across a stated period — three qualifiers the company itself chose, and all three shrink the claim.
Design and operation are separate questions: whether the control was built to do the job, and whether it then did it. A policy answers only the first.
The document says nothing about the time since the period closed, and carries no validity period, because it is not a licence.
Nothing in it ranks the company against anybody; there is no grade and no scale, which is why products get built to invent one.
Read it backwards — boundary, period, exceptions, then description. The first three are short and they decide what the fourth one means.
A short honest description is easier to be right about than a long generous one, so claiming less is the cheaper position.