A department celebrates a lower defect rate after one good week. Another declares failure after a temporary increase in complaints despite stable service conditions. Both judgments may confuse random fluctuation with a changed process. A baseline is not simply last month’s number. It is a defined reference period whose operating conditions, measurement method and underlying variation are understood well enough to support comparisons. The organization must decide what signal would justify action, what evidence would support a claim of improvement and when a new baseline should replace the old one.
Q1What makes a baseline credible?
Select a period with reasonably consistent process scope, collection method and operating conditions. A baseline spanning a system migration or new product introduction can mix fundamentally different processes. Document the event definition, volume, case mix, instrument settings and any known disruptions during the reference period.
A simple arithmetic average may be a useful descriptive summary, but it does not reveal how much a stable process usually varies. Plot the sequence of results and inspect seasonality, outliers and missing observations. Where sample sizes are very small, avoid presenting differences as statistically established improvement. The credibility of the baseline depends on the comparability of its underlying observations.
Baseline question. A reference point is only useful when the process population and observation method are sufficiently stable. Record the conditions that made the original period representative and the conditions that would disqualify it from future comparisons.
Technical context: ISO 7870-2:2023 — Shewhart control charts is one verified source for the concepts discussed here.
Q2Why can an apparently good week be random?
When demand, material mix or operator assignment changes, the apparent defect rate can shift without the process becoming better. Random variation also affects outcomes even under consistent conditions. If managers respond to every short-term movement, they may change settings repeatedly and make a stable process less predictable.
Look for sustained evidence using an appropriate time series or statistical process control technique. ISO 7870-2:2023 provides guidance on Shewhart control charts, but selecting the chart, assumptions and control limits requires statistical competence and a suitable data-generating process. Do not label an arbitrary dashboard target line as a control limit or assume that meeting a target proves statistical stability.
Noise warning. A single favorable week can occur without any meaningful process change. Before celebrating, look at normal dispersion, sample sizes and operating mix. A control limit derived from valid process evidence is different from a desired management target.
Q3How should an unusual signal be investigated?
First check whether collection definitions or source systems changed. Then examine whether the process itself encountered a special cause: equipment failure, supplier change, unusual order mix or shift coverage. A credible investigation links the signal to an observed change rather than assigning a reason because it sounds plausible.
Protect against hindsight. Record what was known at the review date, which hypotheses were tested and what evidence changed the conclusion. Where a special cause has been demonstrated, apply a focused corrective action and watch the subsequent observations. An unexplained outlier should remain visible rather than disappearing from the dataset to improve the trend.
Signal investigation. When a point looks unusual, inspect equipment, materials, staff assignment and data collection changes before declaring a cause. Preserve rejected hypotheses and sample evidence where material. This avoids turning a plausible story into an unsupported finding.
Q4When is rebaselining justified?
Resetting a baseline is warranted when the process or measurement method materially changes and old values cannot support like-for-like comparison. Examples include changed product specifications, a revised definition of first-pass acceptance or a validated automation that captures previously missing events. Merely missing a management target is not a legitimate reason to move the starting line.
Preserve the previous reference period and the approved rationale for the new one. Use a bridging calculation or parallel run if feasible. For improvements that are intended to change the process permanently, wait for enough post-change evidence before treating the new performance range as established. Explain to users whether observed movement reflects process change, population change or redefined measurement.
Rebaseline rule. An updated measurement method can legitimately require a new starting point. Keep the earlier values and record whether any parallel calculation makes the series comparable; never reset simply to make a missed objective disappear.
Q5What separates targets from control limits?
A customer-required response time may be a specification or commitment; an internal goal expresses desired performance; a statistical control limit describes expected process variation under a particular model. These are not interchangeable lines on a chart. Crossing a target may require customer action even if the process is statistically stable.
Explain the different questions separately: is the result acceptable to the customer, is the process predictable and has its performance improved? A process can be stable but systematically incapable of meeting a specification. It can also meet most current targets while showing unusual variation that merits investigation. Management decisions should consider all three dimensions.
Three questions. Ask separately whether the output met a contractual requirement, whether the process was statistically predictable and whether the intervention improved results. A stable process can be incapable, and an apparently successful period can still contain special-cause instability.
Q6How should improvement effectiveness be judged?
Begin with a stated problem and baseline measure, including the uncertainty in measurement. Define the intervention and expected mechanism: reducing setup variation, standardizing a high-risk task or improving input quality. Track implementation as a separate process measure; completion of training is not itself evidence that defect rate has declined.
Compare performance after change with the reference period using a method appropriate to the data and operating conditions. Look for unintended effects such as slower throughput or increased rework. If evidence is weak, describe the result as promising rather than proven. Retain observations that could falsify the improvement hypothesis and plan continued monitoring.
Project proof. Completion of training, maintenance or a new instruction is evidence of the intervention, not proof of its effect. Compare subsequent results on the same accepted measurement basis, then examine side effects such as throughput loss or increased workload.
| Decision domain | Owner or objective | Observable evidence |
|---|---|---|
| Baseline | Comparable operating period | Definitions and observed variation |
| Signal | Unusual process behavior | Time series and cause record |
| Change | Approved intervention | Implementation and control evidence |
| Rebaseline | Material method change | Versioned bridging rationale |
Q7How can managers avoid overreacting to noise?
Select review frequency based on process dynamics and data volume. A daily control chart may help a high-volume production line, while a low-volume incident measure may need a longer aggregation period and case review. An executive report should highlight meaningful signals and their verified causes instead of showing every fluctuation as urgent.
Train reviewers to ask whether the process, measurement system and population remained comparable. Distinguish immediate safety or legal escalation from statistical performance interpretation; the first may require action on a single event. Use ISO 9001:2026 performance evaluation requirements as management-system context, while selecting statistical techniques according to the real decision rather than implying that one chart type is universally mandatory.
Review discipline. For rare events, use case-level evidence and an appropriately long analytical period rather than unstable percentages from tiny denominators. For high-frequency measurements, use suitable control methods and respond to meaningful signals instead of every routine fluctuation.
Practical Review Questions
- Does the reference period use a stable definition and process?
- Are target lines and statistical limits distinguished?
- Has a meaningful observed change been separated from random fluctuation?
- Is rebaselining governed with versioned evidence?
Test an Improvement Claim Against a Defensible Baseline
Define the target outcome and select a reference period in which the measurement method and operating conditions were comparable. Plot the sequence of data, evaluate sample size and ordinary variability and record significant process changes. Where appropriate, obtain competent advice on selecting a statistical control method rather than turning an arbitrary management threshold into a control limit. Retain the original population and case-mix assumptions to prevent accidental comparison of unlike work.
After an intervention, compare later performance using a consistent definition and inspect the proposed causal mechanism. Separate observed outcome improvement from implementation activity and consider adverse trade-offs such as reduced throughput or more rework. If evidence remains ambiguous, state that limitation and continue observation. Record any approved baseline reset and the effect on historic comparisons so the business can challenge the claim in future reviews.
Closing Perspective
A credible improvement claim joins a stable definition, a meaningful reference, appropriate analysis and evidence of the change mechanism. A target crossed once is a prompt to investigate, not a verdict on process capability.
References & Further Reading
Official ISO publications informing the measurement and data-quality method discussed above. Some ISO publications set requirements while others offer guidance or reference models. Select the applicable edition and the evidence appropriate to the organization.