They are testing your description, not your security

Somebody is going to spend a week inside your organisation and then write about one page about you. It is worth knowing, before that week, what the page says.

Stripped of its formality, it says this: here is the description management gave us of their system and the controls in it; in our opinion those controls were suitably designed, and they operated as described, across the stated period.

There are two documents in that sentence and only one of them belongs to the person writing it. The description is yours. The opinion is theirs, and it is an opinion about your description.

Predict

Two organisations each run an access review, and each one genuinely ran it four times last year, with dated records. Organisation A's written description says the review happens monthly. B's says quarterly. Which one is carrying a problem into the audit?

Choose one answer

A — it described twelve and four happenedB — four reviews a year is less evidence than twelve would beNeither — the review ran, which is the thing that mattersBoth — four reviews is thin either way

Lock in answer

Why the scope is that narrow

It is tempting to read the narrowness as evasion — a profession carefully promising as little as it can get away with. Worth resisting, because the narrowness is the only reason the document is worth anything at all.

Consider the alternative. Is this organisation secure? has no boundary, no period, no population and no test. Answering it honestly would require knowing what every attacker knows and what every employee will do next Tuesday. A document asserting it would be worthless precisely because nobody could have checked it.

Did the thing you described as happening quarterly happen quarterly, between these two dates? has a population, a date range and a procedure. It is small, it is checkable, and it is close to the largest claim an outsider can stand behind.

So the exercise tests the small thing on purpose. The buyer's leap — from they described controls and the controls ran to my data is probably alright there — is the buyer's own inference, and it is not in the report.

Something people say

“A clean report means the company is secure.”

Why it spreads

Procurement needs a yes or a no, and a document with a short opinion at the front and no exceptions in it reads exactly like a yes. The report is also long, technical and expensive, and things that are long, technical and expensive get assumed to be thorough in proportion to their cost.

The part that is true

There is a real correlation underneath it. An organisation that can describe its controls accurately, name an owner for each one and produce dated artefacts for a whole period is usually better run than one that cannot. The ability to answer the question is weak evidence about the thing the question sits near.

Where does the inference actually break?

The controls were tested on a sample rather than in fullThe opinion is bounded by a description the organisation wrote itselfThe auditor is paid by the organisation being examinedReports are usually months out of date by the time a buyer reads one

What this changes about your week

Once the object under test is the description, three things stop being mysterious.

Writing the description is the expensive decision, not the paperwork at the end. Every cadence you declare is a population somebody will count. Monthly costs roughly three times what quarterly costs, for as long as you keep saying it, and the only thing that decided it was which sentence you borrowed.

A control that works and is described wrongly is a finding. The work happened. The description said something else. That is a mismatch between two documents, and the cheaper one to change is usually the description — going forward, honestly, with a date on the change.

A control described accurately that did not run is a different finding, and a harder one, because every available option costs time you have already spent. That is lesson eight in this track.

Check

Your description names the head of engineering as the person who performs the quarterly review. For the last nine months her deputy performed it, properly, with dated records and removals. What is that?

Choose one answer

A control failure — the named person did not do itNothing at all — the review happenedA description that no longer matches operationA security weakness, because a deputy has less standing to remove access

Check answer

›Why they make you write the descriptionOptional

It looks like an administrative burden shifted onto the client, and it is not.

An opinion has to be about something specific or nobody can check it, including the next practitioner who reviews the file. If the auditor wrote the description, the same party would be asserting the claim and verifying it, which is the one arrangement independence exists to prevent.

So the asymmetry is deliberate: you assert, they test the assertion. It also means the highest-leverage hour in the whole exercise is the one spent reading your own description as a stranger would, counting the promises in it.

Worth remembering

• The opinion covers two things: whether the described controls were suitably designed, and whether they operated as described, over a stated period.
• The description is written by you, and it sets the boundary of everything that gets tested.
• A well-run organisation that described its controls loosely carries a real problem; an accurate description of modest controls does not carry that particular problem.
• The scope is narrow because narrow is close to the largest claim an outsider can actually check — anything wider would be unverifiable and therefore worthless.
• A mismatch between description and operation is a finding about documents, and the cheaper correction is usually to the description, going forward, with a date on it.
• Nothing in the opinion says the organisation is secure, and the jump from one to the other belongs to whoever reads the report.