A management board sees a site improve from 75 percent to 100 percent on a compliance checklist. Applause follows until someone notices that the earlier period covered four cases and the later period covered three. One event can dominate such a small sample. Percentages are convenient for comparisons, but they discard information about the number, variation and selection of observations that produced them. For ISO management systems, a percentage can be an honest description of recorded cases and still be a poor basis for concluding that a process became more reliable. The reader needs to know what exactly the figure can and cannot establish.
01Always show the numerator and denominator
Present a percentage as an accompanying summary rather than as the whole result. “Three of four inspections acceptable (75%)” provides materially more information than “75% acceptable.” A single additional success would change that first percentage by 25 percentage points. A percentage from 4 cases should not be ranked alongside a percentage from 4,000 transactions without acknowledging the difference in precision and operational exposure.
Define the eligible population before selecting the sample. For a first-pass outcome, record whether all jobs were eligible or only those that reached final inspection. For customer complaints, distinguish complaints per shipped unit from the share of resolved complaints. The denominator is part of the business question, not a technical afterthought. A rate based on selected easy cases can be misleading even when the number of observations is large.
If there are too few events to support a rate, a count plus a brief case narrative may communicate better. Do not suppress a serious failure simply because one event makes a small denominator look dramatic; inspect its consequence separately.
Technical context: ISO 9001:2026 — Quality management systems — Requirements is one verified source for the concepts discussed here.
02Distinguish observed rate from underlying reliability
A sample rate is a description of the observations taken; it is not proof of the long-run process probability. For independent binary outcomes, uncertainty can be expressed through an appropriate interval estimate. At small sample sizes, approximate symmetric intervals can behave poorly, which is why exact binomial methods or other properly chosen methods may be warranted.
NIST statistical guidance explains exact confidence intervals for small numbers of failures or observations. Such intervals rely on assumptions about how data were sampled and the event process. They are not magic corrections for convenience samples, repeated observations of one batch or events whose outcomes depend on each other. State the method and assumptions if uncertainty bounds are used in a management presentation.
Where formal statistical analysis would be disproportionate, a plain-language qualification such as “three observations only; directional, not conclusive” is better than a precision-looking chart. The key is to avoid presenting a descriptive percentage as an established improvement claim.
03Check sampling design before comparing teams
Compare how the observations entered the dataset. One facility may inspect every dispatch while another samples only complaints; a sales team may rate only completed projects while unresolved contracts remain outside the survey. Those percentages are measuring different populations. Differences in product mix, operating shifts and task complexity can likewise change the event mix.
Document whether a sample is random, systematic, risk-based or convenience-based. A targeted inspection of high-risk batches is useful for assurance but not naturally comparable with a representative process sample. Stratify where a major factor changes exposure: urgent service calls, new operators or supplier categories may warrant separate summaries rather than a single weighted average.
Before describing a ranking as meaningful, check that selection rules, data periods and case definitions are reasonably aligned. If not, report the differences explicitly or compare performance within similar subgroups.
04Use examples that reveal denominator sensitivity
Consider a clearly hypothetical service example: one team closes 9 of 10 cases on time, another closes 92 of 100. The first team shows 90 percent and the second 92 percent, but the difference alone is weak evidence of a meaningful performance gap. A single delayed case changes the first result by ten percentage points; the larger sample is much less sensitive to any one transaction.
Likewise, moving from 4 out of 5 to 5 out of 5 produces a 20-point rise without demonstrating that future performance will always be perfect. Avoid drawing a trend line through such sparse data without annotating counts. Small rates can still prompt action, but the action should be guided by event severity, operational knowledge and measurement uncertainty.
When leadership asks for a single number, show both rate and count in the same table. Add a short note on whether the period is comparable with the prior period and how many additional observations would materially change the conclusion.
05Separate safety significance from statistical confidence
Some processes require attention after one adverse event regardless of sample uncertainty: an uncontained hazardous release, an unauthorized product shipment or a critical safeguarding failure. Statistical confidence concerns what can be generalized from observations. It does not determine whether an incident deserves immediate containment under the organization’s risk and legal obligations.
Use consequence-aware review alongside performance summaries. A team with one severe event out of a small population should not wait for a larger denominator before investigating it. Conversely, a few isolated harmless failures may warrant verifying the recording method and process conditions before a company-wide redesign. Decision rights should reflect the risk, not the rhetorical force of the percentage.
Require both numerical context and event detail. This allows leadership to avoid dismissing serious cases as noise while resisting unsupported claims that tiny changes in a rate prove broad process deterioration.
06Apply an appropriate statistical view
Run charts or control charts can be useful when there are sufficient repeated observations, stable definitions and suitable subgroup logic. ISO 7870-2:2023 is specifically concerned with Shewhart control charts; it is not a demand that every ISO KPI be plotted that way. Select a method appropriate to the variable and data structure, and ask a competent analyst to explain why it fits.
For infrequent events, report exposure and elapsed time as well as counts. Aggregating many weeks may improve denominator size but can hide a brief period of severe process instability. Rolling windows improve smoothness at a cost: adjacent values overlap heavily and should not be treated as independent experiments. Make the trade-off visible rather than using whichever graph looks most reassuring.
If sample sizes change substantially, display denominators over time. An apparent stable percentage may hide the loss of observation coverage or a change in the kinds of work completed.
07Make review meetings intellectually honest
In management review, ask what decision would change if the observed rate moved by one event. When that answer is “the entire ranking,” the ranking may not be robust enough for incentives or supplier sanctions. Use a defined process for flagging rates with sparse denominators, and agree how managers will supplement them with process evidence.
A mature reporting pack distinguishes count, measured rate, coverage and interpretation. It labels illustrative numbers when showing training examples. It avoids implying causal effectiveness from two points on a chart. In performance conversations, the most important question is often whether the organization now knows more about the mechanism of failure, not whether the latest percentage is marginally higher.
Sound measurement governance resists false certainty. A small sample should invite better inquiry, not lower accountability or exaggerated celebration.
Practical Review Questions
- Can each published percentage be traced to its count?
- Is the sample representative of the stated process?
- Would one event reverse the decision?
- Are severe cases assessed independently of the rate?
Introduce denominator-aware reporting
Modify one supplier or service scorecard to display successful events, eligible events and the percentage together. Add sampling method and any exclusions in a footnote immediately beside the result, rather than in a hidden method appendix.
Take two past reporting periods and ask leaders whether the original ranking survives when one observation changes. If it does not, change the language of the conclusion and choose a better way to investigate process performance. Add a small-sample review rule to the management pack. For every rate below the agreed observation-count trigger, show the eligible population, numerator and a sentence describing what one additional event would do to the result. Do not impose an arbitrary universal minimum sample as a legal or ISO requirement; select the trigger in light of the decision and the consequences of being wrong. For a safety-critical failure, investigate the event even when inferential precision is poor. For a reward or ranking, defer a definitive comparison until the denominator and case mix support it. Record both decisions separately so the organization does not mistake statistical restraint for a failure to act.
Closing Perspective
Percentages summarize observations, but sample size and sampling design determine how confidently they can support comparisons. Show counts, identify uncertainty and retain judgment about the actual consequences of individual cases.
References & Further Reading
Official ISO publications informing the measurement and data-quality method discussed above. Some ISO publications set requirements while others offer guidance or reference models. Select the applicable edition and the evidence appropriate to the organization.