How Experienced Auditors Select Meaningful Samples

Start With the Decision the Sample Must Support

Before requesting records, state the audit criterion, process boundary and decision at stake. “Does purchasing follow the procedure?” is too broad to guide a useful sample. “For recent high risk purchases, can approved requirements be traced to supplier selection, receipt checks and acceptance?” identifies the control chain to test. Once the question is explicit, the auditor can decide whether an order, supplier, batch, incident or site is the appropriate sampling unit.

Clarify the population and period: Open and closed orders, all shifts, all supplier categories or a defined production window. The population may be fragmented across systems, and completeness of the list itself can be a control question. Ask how the list was generated, what transactions were excluded and whether cancellations, exceptions and manual overrides remain visible.

Audit judgement: a sample should be traceable back to its population, selection rationale and audit objective. A detailed checklist cannot repair a selection that excludes the process conditions most relevant to the conclusion.

To make that decision operational, agree in advance what counts as satisfactory evidence for the sampled control. If the audit question concerns authorization, merely locating a username may be insufficient where responsibilities require approval of a particular class of exception. If the question concerns an inspection, a completed field may not show that the specified characteristic was tested before release. Define these checks before seeing the auditee's favorite records. This protects the selection rationale from being quietly changed to match whatever documents happen to be available and makes follow up discussions with process owners more precise.

Practical preparation: ask the process owner for a complete list of the candidate transactions, not a curated packet, and preserve the list definition. Mark which units are outside scope and why. For an audit of corrective action, decide whether a “case” means one original nonconformity, one corrective action assignment or one verified effectiveness review. The distinction changes the population and the possible conclusion. Have a second auditor examine the population boundary if it is likely to be contested at the closing meeting.

Stratify by Consequence, Variation and Process Change

Risk based selection examines where a failure could matter most: Safety critical services, environmental exposures, sensitive information, customer specific commitments or high value procurements. It also looks for variation. A process that functions well during office hours may depend on different controls on the night shift; a stable central policy may be implemented differently at remote sites.

Build a short sampling map with strata such as product family, transaction value, shift, process owner, automated versus manual route, supplier maturity and period before or after a system change. Select from the relevant strata rather than choosing the first records made available. Previous incidents can inform targeting, but they must not become the only items examined.

Use at least one ordinary, uneventful transaction as a comparison where the audit question warrants it. Investigating exceptions alone is valuable for failure analysis, but it does not show whether the stated routine consistently operates.

When there are too many possible dimensions, prioritize those that could change how the process behaves. For incoming materials, supplier category and receipt shift may matter more than the alphabetic order of document numbers. For a service desk, incident severity, escalation channel and assignment queue may be more informative than the calendar day. Record why one stratum was omitted if it carries a material risk. Auditors rarely have unlimited fieldwork time; the important question is whether the necessary variation was examined well enough for the conclusion actually written.

Combine Targeted and Unpredictable Selection

Judgemental samples are useful when auditors need to examine consequential, unusual or suspect conditions. Random selection from a complete population introduces an element of unpredictability and reduces the ability of an auditee to preselect polished examples. These approaches serve different purposes and can be combined explicitly in an internal audit plan.

Do not label an informal sample “statistically representative.” Statistical sampling needs a defensible design, defined population, assumptions and analysis appropriate to the objective. ISO 19011 provides guidance on audit sampling and evidence; it does not convert a small convenience sample into a quantified conformity rate. Explain when the sample is non statistical and why it is nevertheless fit for the audit task.

If one sample triggers a concern, extend sampling along the relevant failure mechanism rather than mechanically doubling the number of files. The expansion should be aimed at learning whether the anomaly is isolated or systemic.

A practical plan can document a targeted group and a separate unpredictable group without publishing a complicated mathematical claim. Ask a reliable system custodian for a complete export, capture the query definition and select some cases without the process owner first filtering them. In very small populations, reviewing all relevant records may be more efficient; in large populations, a deeper targeted sample may be justified by consequence. The choice is a professional decision, and the working papers should explain it in language a reviewer can understand.

A confidence problem often appears when management shows excellent quarterly performance figures but the data exclude manual interventions. Identify the source events feeding the dashboard, then select cases through both the reported channel and the alternate route. A well chosen contrast between ordinary operations and authorized exceptions may tell the auditor more than ten adjacent transactions from the same automated sequence. This is especially relevant for systems in transition, where temporary workarounds can last longer than the project plan.

Follow Each Selected Case End to End

After selecting a transaction, follow the actual trail. For a customer complaint, inspect intake, classification, assignment, containment, root cause analysis, corrective action and evaluation of effectiveness where these steps apply. Compare timestamps and responsibilities across source systems; ask staff how exceptions are escalated when a required field or approval is missing.

An order to delivery trail might reveal that incoming inspection records are complete but released product can bypass quarantine through an authorized override. Sampling the entire chain makes the exception visible. Sampling only the form called “final inspection checklist” might miss it. Record each transition and which control was expected to operate there.

Beware of “document hopping,” where each new piece of paper leads to another unrelated procedure while the original sampled event disappears from view. Maintain a simple transaction identifier at the top of the working paper. For every handoff, ask which input was used, which control was expected, what state was recorded and which output became available. Where the organization relies on software, check that the particular user role and configuration were in effect at the time. These observations help distinguish an effective control from a well written instruction that was not implemented.

Use Contradictions to Select the Next Sample

The first set of evidence is frequently imperfect. An interview may indicate all supplier changes require approval, while the change log shows a different status. Rather than accepting either version, select the actual change record, user access history, workflow configuration and relevant approval. The next sample is chosen to resolve a specific uncertainty, not to secure agreement with the auditor’s initial view.

Mark missing data explicitly. An unsearchable register, a dashboard that excludes manual work or an audit trail overwritten by routine retention may change what can reasonably be concluded. Lack of accessible evidence is not proof of a failure against every possible criterion; it is a reason to clarify the actual evidence requirement and reporting boundary.

A contradiction can also reveal that the sampling frame itself is unreliable. Consider a closed orders report excluding reopened orders: If reopened orders include many disputed deliveries, choosing only the closed report may systematically miss exactly the risk under audit. Investigate exclusions before expanding the number of randomly selected rows. The auditor can request a reconciliation among operational logs, finance records and management reports, then document any unresolved differences. In this case the next best sample is not necessarily another order; it may be the omitted population and the rule that removed it.

Evidence quality test: for one sampled approval, write down what observable condition would prove the decision took place before the controlled activity and by someone with appropriate authority. Apply the same test to a non approval event. If the second case lacks the necessary control event, check whether the system was designed to record it elsewhere. A comparison with an authorized exception helps the team avoid confusing different workflow routes. The audit sample should challenge control reliability, not simply demonstrate how readily someone can retrieve a favorable screen.

Document the Rationale and Bound the Conclusion

A defensible sampling note records the population, time window, inclusion and exclusion conditions, identifiers examined, reasons for targeted selection and follow up expansion. It distinguishes sampled items from the overall population, and it records the difference between evidence observed, auditee explanation and auditor inference.

Use precise conclusion language. “Three examined exceptions lacked the required approval” is supportable when the criterion and records are identified. “The company never approves exceptions” is not supportable from those three cases. If a process is high risk and the evidence base remains limited, report the limitation and recommend a proportionate follow up.

The reported level of confidence should be understood qualitatively unless the sampling design supports formal inference. State that the sample included different categories and revealed a specific weakness, or that no deviations were found in the defined tests. Avoid numerical expressions such as “99 percent compliant” derived from a handful of chosen records. Good reports also distinguish sample facts from wider exposure: Management may need to perform its own extent review after a finding, with an accountable owner, deadline and evidence of completion.

Illustrative Scenario: A Two Shift Receiving Process

Illustrative scenario: a receiving function operates on two shifts. A risk based sample contains ordinary receipts, two expedited deliveries, a recent rejected lot and a manually corrected supplier record. One late shift receipt shows an electronic accept decision before the required check. The auditor traces the entry to the user, timestamp and disposition authority, then reviews several similar late shift events to understand the possible pattern.

This scenario does not establish any universal sample size rule. It shows a practical sequence: Define the question, vary the selection, trace each case, investigate a contradiction and narrow the conclusion. The most meaningful sample is the one that changes the quality of the audit decision.

The audit team's next action would be to check whether the late shift sequence is an authorized exception, an interface timing problem or a failure in the required control. A single interview may not settle that question. A system event extract, a receiving record and the shift handover log could reveal whether the product was actually released before inspection. If the issue is substantiated, the scope of the resulting finding should follow the confirmed facts, while a broader population check becomes a corrective action or follow up requirement rather than an assumed conclusion.

The closing conversation should include the sampling limit. Explain that selected events came from a defined population over a defined period and that additional cases were selected where material ambiguity arose. If a finding requires management to investigate other shifts or sites, report that requirement as a further extent check, not a conclusion already demonstrated. A clear statement of the population and sampling rationale makes the report useful for later risk decisions and for independent audit programme review.

Closing Perspective

Experienced sampling begins with the question the audit must answer and ends with a bounded conclusion that the selected evidence can actually sustain. Risk, variation, change and exceptions should influence selection; record completeness and source independence should influence confidence. A good auditor does not claim that an unexamined population conforms, nor does an isolated anomaly automatically prove a system wide failure. When the sample exposes a possible common weakness, define the further extent check and its owner. The discipline of transparent sampling makes internal audit findings more useful to managers, future auditors and the people responsible for reliable process performance.