Part IV · Chapter 7 · The Audit · lesson 3
The list arrives, and it is specific
13 min
Builds on Twenty-five clean out of twelve hundred proves less than you think
Part IV · Chapter 7 · The Audit · lesson 3
13 min
Builds on Twenty-five clean out of twelve hundred proves less than you think
The list arrives on a Tuesday, as a spreadsheet, and it is much more specific than you expected.
Twelve rows. Row 1: the quarterly access review for the production database, Q1. Row 2: the same for Q2. Rows 3 and 4, the same for the two remaining quarters. Rows 5 to 9: five named merged changes, by identifier, spread across the period. Row 10: the restore test. Rows 11 and 12: the onboarding records for two named people, both of whom joined in the middle of the period.
Nobody asked for everything. Somebody asked for twelve things, by name, with dates, and the dates are not clustered at the beginning or the end.
Predict
A sample request is three decisions somebody already made.
The population. What the complete set of instances is taken to be — every quarterly review, every merged change, every joiner. If that set came from a list you supplied, how you built the list is now part of the exercise.
The size. How many of them to look at. Twelve from a population of forty is a different claim from twelve from a population of twelve hundred, and track 5 is entirely about the difference.
The selection. Which twelve, and whether you got to choose. Items picked across the period test the cadence. Items you nominated test whatever you were comfortable nominating.
Your response has one job: produce the named artefact, as the system holds it, with the one line of context that says what it is and what it shows.
Here is the uncomfortable version, which is the normal version. Of the twelve rows, nine are a clean export you can produce this afternoon. Two sit in a system whose retention you are not certain reaches back far enough. And one — the Q1 review — you are fairly confident never happened, because the person who performed it was on parental leave that month and nobody picked it up.
What would you do
One row of that request deserves a second look: the five merged changes, selected from a list of all merged changes in the period. Who produced that list?
If it came from the repository, the selection is from the real population and the sample means what sampling means. If somebody exported it by hand, filtered out the infrastructure changes because those go through a different route, and sent the rest, then the population is a judgement call and the interesting question is no longer which five were picked — it is what the filter removed.
That question is not an accusation and it is one you should ask yourself first, because the person building the list is usually the person who knows the exceptions and is trying to be helpful.
Check