Gage R&R False Rejection Process Capability Case Study

A process can look incapable because the measurement system is incapable. When the inspection fixture adds enough repeatability and reproducibility error, conforming parts can be rejected, capability indices can look worse than the actual process, and engineers may adjust a stable machine in the wrong direction.

This case study follows a precision-machined housing bore that was placed on containment after inspection reported a sudden rise in oversize rejects. The first reaction was to change CNC offsets and increase sorting. A measurement-system study showed a different cause: the inspection nest had worn, operators seated parts inconsistently, and the Gage R&R spread consumed most of the tolerance.

The purpose is to connect measurement-system variation, apparent capability, corrected process variation, false rejection, guard bands, containment, cost, and release evidence.

Case Context

The part is an aluminum housing with a precision bore used to locate a bearing sleeve. The bore specification is:

The plant ships about:

A final inspection station uses a bench fixture with a spring-loaded probe. Over two weeks, the measured reject rate rises, mostly from parts reading above the upper specification limit. Production wants to adjust the CNC offset. Quality engineering pauses the adjustment because the fixture has not had a recent measurement-system study.

Item	Value
Lower specification limit, LSL	11.950\ \text{mm}
Upper specification limit, USL	12.050\ \text{mm}
Tolerance width, T	0.100\ \text{mm}
Observed measurement mean, \bar{x}_{obs}	12.004\ \text{mm}
Observed measurement standard deviation, s_{obs}	0.018\ \text{mm}
Old Gage R&R standard deviation, s_{GRR}	0.014\ \text{mm}
Monthly production	18{,}000\ \text{parts}
Rework cost per rejected part	\32$
Scrap cost per rejected part	\85$

The engineering question is:

Is the bore process actually incapable, or is the inspection system creating a false rejection signal?

Initial Evidence

The early evidence is ambiguous.

Evidence	Observation
inspection log	measured rejects increased from about 0.2\% to about 0.7\%
CNC trend	tool offsets and tool life records did not show a matching step change
part rechecks	several rejected parts passed when measured on a reference CMM
operator observation	parts could rock slightly in the bench fixture nest
fixture audit	datum pad showed wear and chips could remain under one seating face
customer feedback	no matching field or assembly complaint had appeared

The process may still need improvement, but the measurement evidence is not credible enough to drive a machine-offset change.

Step 1: Tolerance Width

The tolerance width is:

Substitute:

Engineering Comment

The measurement system must be small compared with this tolerance. If measurement variation is a large fraction of 0.100\ \text{mm}, inspection will create false rejects, false accepts, unstable capability calculations, and disputes between production and quality.

Step 2: Gage R&R Percent of Tolerance

A common screening metric uses six standard deviations of the measurement-system study:

Percent of tolerance consumed by the measurement system is:

Therefore:

Engineering Comment

An 84 percent measurement-system burden is not acceptable for release decisions. The fixture is not merely a little noisy. It is large enough to dominate the inspection result. Any capability number computed from this inspection station should be treated as suspect until the measurement system is corrected or replaced.

Step 3: Apparent Process Capability

If the observed measurements are treated as true part variation, potential capability appears poor:

Substitute:

Upper-side capability is:

Lower-side capability is:

Therefore:

Engineering Comment

This apparent C_{pk} would normally justify containment and corrective action. It does not justify a machine adjustment yet, because the variation includes both true part variation and measurement-system variation.

Step 4: Separate Measurement Variation from Part Variation

When measurement error and part variation are independent, a first-pass variance relationship is:

Solve for estimated true part variation:

Substitute:

Now estimate corrected capability using the same mean:

Upper-side corrected capability:

Lower-side corrected capability:

So:

Engineering Comment

The corrected estimate suggests that the actual machining process may meet a C_{pk}\geq1.33 release target. This estimate is not a release by itself. It is a diagnostic result telling the team to fix the measurement system before changing the process.

Step 5: False Rejection of Near-Limit Parts

Consider a true conforming part with:

It is inside the upper specification limit:

The margin to the upper limit is:

With old measurement-system standard deviation:

the z-score for a false upper rejection is:

The probability that measurement noise alone pushes this true conforming part above the upper limit is approximately:

or about:

Engineering Comment

This does not mean every part has a 24 percent false-reject risk. Parts near the center have much lower risk. It means the old fixture is unreliable exactly where the decision matters most: near the specification limit.

Step 6: Monthly Cost Screen

Using the measured distribution, the observed upper-tail reject probability can be screened with:

A normal tail at z=2.56 is about:

The lower-tail screen is:

with tail probability about:

Total measured reject probability is approximately:

Monthly measured rejects:

If all are reworked:

If all are scrapped:

Engineering Comment

This screen does not prove the exact number of false rejects. It shows that the economic consequence is large enough to justify immediate containment and measurement-system repair. A noisy inspection station can create real production cost even when the part-making process is acceptable.

Step 7: Guard-Band Consequence

If the old measurement system is retained and a conservative expanded uncertainty is estimated as:

then a guarded acceptance interval would be:

Lower guarded limit:

Upper guarded limit:

The guarded acceptance width becomes:

This is only:

of the drawing tolerance.

Engineering Comment

The guard band protects against false acceptance, but it would reject many good parts because the measurement system is too weak. The correct response is not to argue about the guard band. The correct response is to improve the measurement system so the acceptance decision is technically usable.

Corrective Measurement-System Actions

The team corrected the inspection system before moving the CNC process.

Weakness	Corrective action
worn datum pad	replace nest and add wear inspection interval
chips trapped under part	add air blast and visual seating check
operator seating force variation	add hard stop and standard work instruction
probe side load	realign probe perpendicular to bore datum
temperature drift	require part and fixture stabilization before release study
ambiguous reaction rule	require reference CMM confirmation before machine-offset change

After correction, the measurement-system study estimated:

The new percent tolerance is:

Engineering Comment

An 18 percent value is not perfect, but it is acceptable for the stated release criterion and far better than the original 84 percent. Critical characteristics may require an even stronger system depending on safety, customer requirements, and false-accept risk.

Validation with Corrected Measurement

A validation run using the corrected fixture and reference CMM checks produced:

Parameter	Value
validation sample size	150 parts
validation mean, \bar{x}_{new}	12.002\ \text{mm}
observed standard deviation with corrected gage, s_{obs,new}	0.0118\ \text{mm}
corrected Gage R&R standard deviation, s_{GRR,new}	0.0030\ \text{mm}

Estimate part variation:

Compute corrected capability:

Upper-side capability:

Lower-side capability:

Therefore:

Engineering Comment

The corrected evidence supports release if the process is stable and the sample represents normal production. The team should still monitor centering because the process is slightly above nominal. The key decision is that machine offset should not be changed based on the old fixture data.

Release Decision

The release decision was:

1. keep affected lots on hold until remeasured by the corrected fixture or reference CMM;
2. release conforming inventory only with traceable measurement records;
3. do not change CNC offsets based on old fixture data;
4. update the control plan with the new fixture, seating method, air blast, and reaction rule;
5. require a new Gage R&R study after fixture maintenance, operator change, or method change;
6. review the first four normal production weeks for measured reject rate, Cpk, fixture wear, and customer escapes.

The release criterion is:

with:

and no unexplained difference between production fixture and reference CMM.

Risk Review

Before correction, the system could drive unnecessary process changes, reject good parts, and still risk accepting a bad part because the inspection evidence was noisy.

Failure mode	Severity	Occurrence	Detection	RPN
old fixture drives false rejection and wrong process adjustment	6	5	7	6(5)(7)=210
corrected fixture with Gage R&R, CMM check, and reaction rule	6	2	2	6(2)(2)=24

Severity remains moderate because customer escape and production disruption are both consequential. The improvement comes from reducing occurrence through fixture control and improving detection through measurement-system studies and reference checks.

Common Mistakes

• Treating measured variation as process variation without checking the measurement system.
• Adjusting a machine because Cpk looks low when Gage R&R is unacceptable.
• Using inspection sorting as a permanent substitute for measurement-system repair.
• Ignoring false rejection because it seems safer than false acceptance.
• Reporting capability to two decimal places when the gage consumes most of the tolerance.
• Running a special clean validation instead of normal production with real operators, fixtures, material, and temperature.
• Closing corrective action after replacing parts without updating the control plan and reaction rules.

Transferable Lesson

Process capability is only as credible as the measurement system behind it. A low C_{pk} can mean the process is poor, the mean is off-center, the distribution is unstable, the sample is biased, or the inspection system is adding noise.

The practical engineering sequence is:

1. hold suspect product when the consequence of escape is material;
2. verify measurement-system adequacy before changing the process;
3. separate measurement variance from part variance where the assumptions are valid;
4. confirm near-limit parts with a reference method;
5. repair fixtures, methods, calibration, operator instructions, and environmental controls;
6. recalculate capability only after measurement evidence is credible;
7. update the control plan so the same false signal does not recur.

The safest quality decision is not always more inspection. Often it is better measurement.