Study guide

9.3 Measurement System Analysis: Bias, Linearity, and Stability

ASQ Certified Quality Technician CQT

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Quality Philosophies, Principles, and Management Systems

1.1 Quality Philosophies & Pioneers 1.2 Quality Management Systems, Standards & Documentation 1.3 Customer-Supplier Relationships & Quality Culture

2 Chapter 2: Cost of Quality, Teams, and Professional Ethics

2.1 Cost of Quality (COQ) & Economic Trade-Offs 2.2 Quality Improvement Teams & Meeting Management 2.3 Professional Conduct & ASQ Code of Ethics

3 Chapter 3: Basic Quality Tools & Continuous Improvement

3.1 Process Mapping, Flowcharts, and SIPOC 3.2 Cause-and-Effect Analysis & Check Sheets 3.3 Pareto Analysis & Prioritization 3.4 Histograms & Scatter Diagrams

4 Chapter 4: Descriptive Statistics and Probability Distributions

4.1 Data Types, Sampling Methods, and Populations 4.2 Measures of Central Tendency and Dispersion 4.3 Normal Distribution & Z-Score Calculations 4.4 Discrete Distributions: Binomial, Poisson, and Hypergeometric

5 Chapter 5: Statistical Process Control (SPC) Fundamentals & Variables Charts

5.1 SPC Fundamentals & Process Variation 5.2 Control Limits vs. Specification Limits 5.3 X-bar and R Charts 5.4 X-bar & s Charts and Individual-Moving Range (I-MR) Charts

6 Chapter 6: Attributes Control Charts and Out-of-Control Patterns

6.1 Control Charts for Nonconforming Units: p and np Charts 6.2 Control Charts for Nonconformities: c and u Charts 6.3 Out-of-Control Rules & Process Analysis

7 Chapter 7: Process Capability and Performance Analysis

7.1 Process Stability Prerequisite & Capability Basics 7.2 Potential Capability (Cp) and Actual Capability (Cpk) 7.3 Process Performance (Pp, Ppk) and Long-Term vs. Short-Term Variation 7.4 One-Sided Tolerances and PPM Defect Calculations

8 Chapter 8: Metrology Fundamentals, Standards, and Calibration

8.1 Metrology Foundations & Environmental Controls 8.2 Precision Gage Blocks & Standards Hierarchy 8.3 Calibration Systems & NIST Traceability 8.4 Out-of-Calibration Procedures & Reverse Traceability 8.5 M&TE Identification, Control, and Customer-Supplied Equipment

9 Chapter 9: Measurement Instruments and Measurement System Analysis (Gage R&R)

9.1 Common Dimensional Hand Measuring Instruments 9.2 Fixed Gages, Optical Comparators, and CMM Systems 9.3 Measurement System Analysis: Bias, Linearity, and Stability 9.4 Variable Gage R&R: Repeatability and Reproducibility 9.5 Electronic, Weighing, and Force Measurement Equipment

10 Chapter 10: Technical Drawings, Tolerancing, and GD&T (ASME Y14.5)

10.1 Engineering Drawing Basics & Projection Methods 10.2 Datums and the Datum Reference Frame 10.3 Form and Orientation Controls in GD&T 10.4 Location, Profile, and Runout Controls with Modifiers 10.5 Classification of Product Characteristics and Defects

11 Chapter 11: Inspection Planning, Sampling Plans, and Material Disposition

11.1 Inspection Planning & Inspection Points 11.2 Acceptance Sampling & ANSI/ASQ Z1.4 11.3 Switching Rules & Variables Sampling (Z1.9) 11.4 Nonconforming Material Control & MRB Dispositions 11.5 Types of Measurements, Unit Conversion, and Rounding Rules

12 Chapter 12: Quality Audits

12.1 Quality Auditing Principles & Types 12.2 The Audit Process: Planning, Performance, and Reporting 12.3 Audit Tools and Techniques

13 Chapter 13: Risk Management, Corrective and Preventive Action, and Safety

13.1 Risk Assessment Inputs, Monitoring, and Control Plans 13.2 FMEA, Action Priority, and Technician Workplace Safety 13.3 Root Cause Analysis & Problem-Solving 13.4 Corrective and Preventive Action (CAPA) & Verification 13.5 Preventive Action, Mistake-Proofing, and Read-Across

9.3 Measurement System Analysis: Bias, Linearity, and Stability

Key Takeaways Explain This Topic AI

Measurement System Analysis (MSA) partitions observed total process variation into actual part-to-part variation and measurement system error (sigma^2_total = sigma^2_part + sigma^2_gage), ensuring quality decisions are not compromised by gage error.

Accuracy (trueness) encompasses the location components of measurement error—bias, linearity, and stability—while precision encompasses the spread components (repeatability and reproducibility).

Bias represents the systematic difference between the observed measurement average and a certified reference master value, evaluated for statistical significance using a one-sample t-test at alpha = 0.05.

Linearity measures how bias changes across the operating range of the gage, quantified by the slope (b) of the best-fit linear regression line of bias versus reference standard values.

Stability tracks measurement system drift over calendar time by charting periodic measurements of a reference standard on an SPC control chart (X-bar & R or I-MR), where out-of-control signals detect calibration decay.

Last updated: September 2026

  • 9.3 Measurement System Analysis: Bias, Linearity, and Stability
  • The Purpose of Measurement System Analysis (MSA) and AIAG Standards

In statistical quality control, data drives every critical decision: accepting or rejecting lots, computing capability indices ( C p , C p k C_p, C_{pk} C p ​ , C p k ​ ), tuning machine offsets, and declaring process stability. However, the numerical value registered by an inspector or automated gage is rarely the exact true value of the part. Every measurement contains an inherent degree of measurement error.

If measurement error is large relative to part tolerance or process variation, the manufacturing team operates under a false reality:

Producer's Risk ( α \alpha α Error / False Alarm) : Conforming parts near specification limits are misclassified as nonconforming due to gage error, leading to unnecessary scrapping, reworking, and downtime.

Consumer's Risk ( β \beta β Error / Miss) : Nonconforming parts are erroneously accepted and shipped to customers because gage inaccuracies mask out-of-spec dimensions.

To quantify and control this error, the Automotive Industry Action Group (AIAG) published the widely adopted Measurement System Analysis (MSA) Reference Manual (currently in its 4th Edition). Under MSA principles, the observed total process variation ( σ Total 2 \sigma^2_{\text{Total}} σ Total 2 ​ ) is the mathematical sum of actual part-to-part manufacturing variation ( σ Part 2 \sigma^2_{\text{Part}} σ Part 2 ​ ) and measurement system variation ( σ MS 2 \sigma^2_{\text{MS}} σ MS 2 ​ ): σ Total Observed 2 = σ Part-to-Part 2 + σ Measurement System 2 \sigma^2_{\text{Total Observed}} = \sigma^2_{\text{Part-to-Part}} + \sigma^2_{\text{Measurement System}} σ Total Observed 2 ​ = σ Part-to-Part 2 ​ + σ Measurement System 2 ​ σ Total Observed = σ Part 2 + σ MS 2 \sigma_{\text{Total Observed}} = \sqrt{\sigma^2_{\text{Part}} + \sigma^2_{\text{MS}}} σ Total Observed ​ = σ Part 2 ​ + σ MS 2 ​ ​

MSA evaluates whether the measurement system is capable of detecting process shifts and sorting good product from bad product before the data is used in quality decisions.

Accuracy (Trueness) versus Precision (Variation)

A cornerstone of the ASQ CQT body of knowledge is the rigorous distinction between accuracy (trueness) and precision (variation) : ACCURACY (LOCATION) VS. PRECISION (SPREAD):

  • Target 1: Accurate & Precise Target 2: Precise but BIASED (Inaccurate)
  • +-------+ +-------+
  • | * | | |
  • | *** | (Tight cluster on bullseye) | | *** (Tight cluster, offset)
  • | * | | | *** from bullseye!)
  • +-------+ +-------+
  • Target 3: Accurate but IMPRECISE Target 4: NEITHER Accurate NOR Precise
  • +-------+ +-------+
  • | * * | | * |
  • | * | (Centered average, but | * | (Scattered wide, off-center)
  • | * * | wide scattered spread!) | * |
  • +-------+ +-------+

Accuracy (Trueness / Location Error) : The closeness of agreement between the average of an infinite number of observed measurement results and a certified reference master value . Accuracy tells you where the measurement distribution is centered .

The three location components of measurement error are Bias , Linearity , and Stability .

Precision (Variation / Spread Error) : The closeness of agreement between independent measurement results obtained under stipulated experimental conditions. Precision reflects the spread or dispersion of the measurement distribution , completely independent of whether the measurements are near the true target.

The two precision components of measurement error are Repeatability and Reproducibility (detailed in Section 9.4).

Measurement Attribute Error Category Core Question Answered Error Components Evaluated

Accuracy (Trueness) Location Error Is the measurement centered on the true standard value? Bias , Linearity , Stability

Precision (Variation) Spread Error Do repeated measurements cluster tightly together? Repeatability (EV) , Reproducibility (AV)

Bias: Systematic Measurement Error and Statistical Significance

Bias is the systematic error or difference between the observed average of repeated measurements and the accepted, certified reference master value of the measured characteristic. Bias = X ˉ − X ref \text{Bias} = \bar{X} - X_{\text{ref}} Bias = X ˉ − X ref ​

Where X ˉ \bar{X} X ˉ is the arithmetic mean of repeated measurements, and X ref X_{\text{ref}} X ref ​ is the certified reference master value (established by a calibration laboratory using NIST-traceable gage blocks or primary standards).

Bias Study Procedure (AIAG MSA Protocol)

Sample Selection : Obtain one representative production part or standard whose critical characteristic has been independently measured in a controlled calibration laboratory to establish the reference master value ( X ref X_{\text{ref}} X ref ​ ).

Execution : Have a single appraiser measure the master part n ≥ 10 n \ge 10 n ≥ 10 to 15 times (conventionally n = 15 n = 15 n = 15 or n = 20 n = 20 n = 20 ) under normal shop-floor operating conditions. The trials must be conducted in random order or separated by intervening parts to prevent operator memory bias.

Calculate Sample Mean ( X ˉ \bar{X} X ˉ ) : X ˉ = ∑ i = 1 n X i n \bar{X} = \frac{\sum_{i=1}^n X_i}{n} X ˉ = n ∑ i = 1 n ​ X i ​ ​

Calculate Sample Standard Deviation ( s s s ) : s = ∑ i = 1 n ( X i − X ˉ ) 2 n − 1 s = \sqrt{\frac{\sum_{i=1}^n (X_i - \bar{X})^2}{n - 1}} s = n − 1 ∑ i = 1 n ​ ( X i ​ − X ˉ ) 2 ​ ​

Calculate Bias : Bias = X ˉ − X ref \text{Bias} = \bar{X} - X_{\text{ref}} Bias = X ˉ − X ref ​

Calculate Percentage Bias : % Bias = ∣ Bias ∣ Tolerance × 100 % ( or ∣ Bias ∣ 6 σ process × 100 % ) \% \text{Bias} = \frac{|\text{Bias}|}{\text{Tolerance}} \times 100\% \quad \left(\text{or } \frac{|\text{Bias}|}{6\sigma_{\text{process}}} \times 100\%\right) % Bias = Tolerance ∣ Bias ∣ ​ × 100% ( or 6 σ process ​ ∣ Bias ∣ ​ × 100% )

Testing for Statistical Significance: The One-Sample t t t -Test

A critical question for quality technicians is: Is the calculated bias a true systematic error, or is it merely random sampling noise?

To answer this, MSA conducts a two-tailed one-sample t t t -test :

Null Hypothesis ( H 0 H_0 H 0 ​ ) : Bias = 0 \text{Bias} = 0 Bias = 0 (the measurement system has zero true bias, μ = X ref \mu = X_{\text{ref}} μ = X ref ​ ).

Alternative Hypothesis ( H 1 H_1 H 1 ​ ) : Bias ≠ 0 \text{Bias} \ne 0 Bias  = 0 (a statistically significant non-zero bias exists).

The t t t -Statistic Formula: t calc = X ˉ − X ref s / n = Bias s X ˉ t_{\text{calc}} = \frac{\bar{X} - X_{\text{ref}}}{s / \sqrt{n}} = \frac{\text{Bias}}{s_{\bar{X}}} t calc ​ = s / n ​ X ˉ − X ref ​ ​ = s X ˉ ​ Bias ​

Where s X ˉ = s n s_{\bar{X}} = \frac{s}{\sqrt{n}} s X ˉ ​ = n ​ s ​ is the standard error of the mean, and the degrees of freedom is d f = n − 1 df = n - 1 df = n − 1 .

Decision Rule (at α = 0.05 \alpha = 0.05 α = 0.05 ):

Look up the critical value t α / 2 , n − 1 t_{\alpha/2, n-1} t α /2 , n − 1 ​ in a Student's t t t -distribution table (for example, for n = 15 , d f = 14 n = 15, df = 14 n = 15 , df = 14 , two-tailed t 0.025 , 14 = 2.145 t_{0.025, 14} = 2.145 t 0.025 , 14 ​ = 2.145 ).

If ∣ t calc ∣ > t crit |t_{\text{calc}}| > t_{\text{crit}} ∣ t calc ​ ∣ > t crit ​ (or if the p p p -value < 0.05 < 0.05 < 0.05 ), reject the null hypothesis . The bias is statistically significant!

Alternatively, compute the 95 % 95\% 95% Confidence Interval for Bias: Confidence Interval = Bias ± ( t α / 2 , n − 1 × s n ) \text{Confidence Interval} = \text{Bias} \pm \left(t_{\alpha/2, n-1} \times \frac{s}{\sqrt{n}}\right) Confidence Interval = Bias ± ( t α /2 , n − 1 ​ × n ​ s ​ ) If the confidence interval does not contain zero , the bias is statistically significant. The gage must be physically recalibrated, mechanically adjusted, or programmed with a software offset compensation factor.

Linearity: Evaluating Bias Across the Operating Range

A gage may exhibit zero bias when measuring a 1.000 " 1.000" 1.000" standard, but develop an unacceptable + 0.003 " +0.003" + 0.003" bias when measuring a 5.000 " 5.000" 5.000" part. This phenomenon is evaluated through Linearity .

Important

Linearity : The difference in bias values across the expected operating range of the gage. It measures whether the instrument's accuracy remains constant or degrades as part sizes increase or decrease. LINEARITY REGRESSION PLOT (BIAS VS. REFERENCE VALUE):

  • Bias (Y-axis)
  • ^
  • | * (Part 5: Large positive bias)
  • | * (Part 4)
  • | * (Part 3: Bias ~ 0)
  • 0 -+--------------*-------------------------------------> Reference Master Size (X-axis)

| * (Part 2) Slope b != 0 indicates Linearity Error!

  • | (Part 1: Negative bias)
  • |
  • Common Physical Causes of Linearity Error

Lead Screw Pitch Wear : Progressive lead error in the precision screw of a micrometer or height gage.

Optical / Capacitive Scale Distortion : Non-linear spacing or warping along the glass encoder scale of a digital caliper or CMM.

Linkage Geometry Deflection : Wear or non-proportional lever movement in dial indicator gear trains.

Electronic Amplifier Gain Error : Non-linear voltage amplification in LVDT electronic bore gages.

Linearity Study Protocol

Select Reference Parts : Choose g ≥ 5 g \ge 5 g ≥ 5 parts (conventionally 5 parts) whose dimensions span the entire operating range of the gage (e.g., 1.000 " , 2.000 " , 3.000 " , 4.000 " , 5.000 " 1.000", 2.000", 3.000", 4.000", 5.000" 1.000" , 2.000" , 3.000" , 4.000" , 5.000" ).

Establish Reference Values : Measure each part in a certified calibration laboratory to determine exact master values: X ref , 1 , X ref , 2 , … , X ref , 5 X_{\text{ref}, 1}, X_{\text{ref}, 2}, \dots, X_{\text{ref}, 5} X ref , 1 ​ , X ref , 2 ​ , … , X ref , 5 ​ .

Conduct Measurements : Have one technician measure each part m ≥ 10 m \ge 10 m ≥ 10 times in completely randomized order.

Calculate Average Bias per Part : B ˉ i = X ˉ i − X ref , i \bar{B}_i = \bar{X}_i - X_{\text{ref}, i} B ˉ i ​ = X ˉ i ​ − X ref , i ​

Perform Simple Linear Regression :

Fit a straight line modeling Bias ( Y Y Y ) as a function of the Reference Value ( X X X ): Y ^ = b X + a ⟹ Bias = b × ( X ref ) + a \hat{Y} = b X + a \quad \implies \quad \text{Bias} = b \times (X_{\text{ref}}) + a Y ^ = b X + a ⟹ Bias = b × ( X ref ​ ) + a Where b b b is the regression slope, and a a a is the Y Y Y -intercept.

Calculating Regression Parameters: b = ∑ ( X i − X ˉ ) ( Y i − Y ˉ ) ∑ ( X i − X ˉ ) 2 b = \frac{\sum (X_i - \bar{X})(Y_i - \bar{Y})}{\sum (X_i - \bar{X})^2} b = ∑ ( X i ​ − X ˉ ) 2 ∑ ( X i ​ − X ˉ ) ( Y i ​ − Y ˉ ) ​ a = Y ˉ − b X ˉ a = \bar{Y} - b \bar{X} a = Y ˉ − b X ˉ

Evaluating Linearity Metrics:

Slope ( b b b ) : Reflects the rate of change of bias per unit increase in measured length.

Percentage Linearity : % Linearity = ∣ b ∣ × Process Variation (or Tolerance) Process Variation (or Tolerance) × 100 % = ∣ b ∣ × 100 % \% \text{Linearity} = \frac{|b| \times \text{Process Variation (or Tolerance)}}{\text{Process Variation (or Tolerance)}} \times 100\% = |b| \times 100\% % Linearity = Process Variation (or Tolerance) ∣ b ∣ × Process Variation (or Tolerance) ​ × 100% = ∣ b ∣ × 100%

Coefficient of Determination ( R 2 R^2 R 2 ) : Measures the proportion of bias variation explained by the linear relationship with part size.

Statistical Significance : A t t t -test is conducted on the slope ( H 0 : b = 0 H_0: b = 0 H 0 ​ : b = 0 ). If p < 0.05 p < 0.05 p < 0.05 , the linearity error is statistically significant, indicating that the gage cannot be corrected by a single constant offset and requires mechanical reconditioning or multi-point calibration mapping.

Stability: Monitoring Measurement Drift Over Calendar Time

A measurement system may demonstrate excellent accuracy and precision during initial qualification. However, over weeks and months of shop-floor operation, environmental temperature swings, component aging, dirt accumulation, and mechanical wear cause the system to drift. This time-dependent variation is evaluated through Stability .

Important

Stability (Consistency) : The total variation in the measurements obtained with a measurement system on the same master standard or parts when measuring a single characteristic over an extended period of calendar time . STABILITY CONTROL CHART (I-MR CHART ON REFERENCE MASTER):

  • Master Reading
  • ^
  • | UCL = 2.0006" - - - - - - - - - - - - - - - - - - - - - - - - - -
  • | * * * (DRIFT!)
  • | CL = 2.0000" -------------*-------*-------*----*------------- (Shift detected)
  • | * *
  • | LCL = 1.9994" - - - - - - - - - - - - - - - - - - - - - - - - - -
  • +-------------------------------------------------------------------> Calendar Time
  • Week 1 Week 2 Week 3 Week 4
  • Stability Study Protocol

Designate a Reference Standard : Dedicate a certified master part or gage block stack specifically for stability monitoring.

Establish Sampling Frequency : Measure the reference standard periodically throughout production (e.g., once or twice per shift, daily, or weekly).

Plot on SPC Control Charts :

If subgroups of repeated measurements ( n = 3 n = 3 n = 3 to 5) are taken: plot on an X ˉ − R \bar{X}-R X ˉ − R control chart .

If a single reading is taken at each time interval: plot on an Individuals and Moving Range ( I − M R I-MR I − MR ) control chart .

Evaluate Out-of-Control Rules :

Points beyond control limits ( ± 3 σ \pm 3\sigma ± 3 σ ) : Signals an abrupt assignable cause (e.g., chipped contact stylus, dropped gage, cracked optical mirror).

Trends (6 or 7 consecutive points continuously increasing or decreasing) : Signals progressive wear, calibration drift, or gradual electronic degradation.

Runs (8 consecutive points on one side of the centerline) : Signals an uncorrected shift in the baseline zero datum or thermal normalization failure.

Shop-Floor Environmental Control and Drift Prevention

To preserve measurement stability, quality technicians must enforce strict shop-floor metrological controls:

The Standard Reference Temperature (ISO 1) : International standard reference temperature for dimensional metrology is 20 ∘ C 20^\circ\text{C} 2 0 ∘ C ( 68 ∘ F 68^\circ\text{F} 6 8 ∘ F ) . Inspection of precision parts with tight tolerances must occur in a temperature-controlled environment.

Thermal Expansion Differential : Different materials expand at different rates: Δ L = L × α × Δ T \Delta L = L \times \alpha \times \Delta T Δ L = L × α × Δ T Where α \alpha α is the coefficient of thermal expansion (aluminum ≈ 13 × 10 − 6 / ∘ F \approx 13 \times 10^{-6}/^\circ\text{F} ≈ 13 × 1 0 − 6 / ∘ F , tool steel ≈ 6.5 × 10 − 6 / ∘ F \approx 6.5 \times 10^{-6}/^\circ\text{F} ≈ 6.5 × 1 0 − 6 / ∘ F , granite ≈ 3.5 × 10 − 6 / ∘ F \approx 3.5 \times 10^{-6}/^\circ\text{F} ≈ 3.5 × 1 0 − 6 / ∘ F ). If an aluminum aerospace housing is machined at 85 ∘ F 85^\circ\text{F} 8 5 ∘ F and inspected against steel gage blocks at 68 ∘ F 68^\circ\text{F} 6 8 ∘ F , thermal shrinkage will cause catastrophic rejection unless temperature compensation is applied!

Thermal Soaking Time : Workpieces and gage blocks brought from an unconditioned warehouse must be allowed to normalize (soak) in the inspection room—typically 1 hour per inch of cross-sectional thickness .

Cleanliness and Air Quality : Air gages rely on clean, dried, oil-free compressed air. Oil mist or desiccant breakdown will clog pneumatic orifices, creating severe stability drift.

  • Step-by-Step Worked Numerical Examples
  • Worked Example 1: Full Bias Study with One-Sample t t t -Test

Scenario : A quality technician conducts a bias study on a digital micrometer used to inspect ground valve stems. A certified reference master gage block certified at X ref = 0.50000 " X_{\text{ref}} = 0.50000" X ref ​ = 0.50000" is measured n = 15 n = 15 n = 15 times by one technician under production conditions. The valve stem blueprint tolerance is 0.5000 " ± 0.0020 " 0.5000" \pm 0.0020" 0.5000" ± 0.0020" (Total Tolerance = 0.0040 " = 0.0040" = 0.0040" ).

The 15 observed readings are:

0.5004 " , 0.5003 " , 0.5005 " , 0.5002 " , 0.5004 " , 0.5006 " , 0.5003 " , 0.5004 " , 0.5005 " , 0.5003 " , 0.5004 " , 0.5002 " , 0.5005 " , 0.5004 " , 0.5006 " 0.5004", 0.5003", 0.5005", 0.5002", 0.5004", 0.5006", 0.5003", 0.5004", 0.5005", 0.5003", 0.5004", 0.5002", 0.5005", 0.5004", 0.5006" 0.5004" , 0.5003" , 0.5005" , 0.5002" , 0.5004" , 0.5006" , 0.5003" , 0.5004" , 0.5005" , 0.5003" , 0.5004" , 0.5002" , 0.5005" , 0.5004" , 0.5006" .

Step 1: Calculate Sample Mean ( X ˉ \bar{X} X ˉ ) ∑ X i = 7.5060 " \sum X_i = 7.5060" ∑ X i ​ = 7.5060" X ˉ = 7.5060 " 15 = 0.50040 " \bar{X} = \frac{7.5060"}{15} = 0.50040" X ˉ = 15 7.5060" ​ = 0.50040"

Step 2: Calculate Sample Standard Deviation ( s s s ) ∑ ( X i − X ˉ ) 2 = 0.000000200 " \sum (X_i - \bar{X})^2 = 0.000000200" ∑ ( X i ​ − X ˉ ) 2 = 0.000000200" s = 0.000000200 15 − 1 = 0.000000200 14 = 0.000000014286 ≈ 0.0001195 " s = \sqrt{\frac{0.000000200}{15 - 1}} = \sqrt{\frac{0.000000200}{14}} = \sqrt{0.000000014286} \approx 0.0001195" s = 15 − 1 0.000000200 ​ ​ = 14 0.000000200 ​ ​ = 0.000000014286 ​ ≈ 0.0001195"

Step 3: Calculate Bias and Percentage Bias Bias = X ˉ − X ref = 0.50040 " − 0.50000 " = + 0.00040 " \text{Bias} = \bar{X} - X_{\text{ref}} = 0.50040" - 0.50000" = \mathbf{+0.00040"} Bias = X ˉ − X ref ​ = 0.50040" − 0.50000" = + 0.00040" % Bias = ∣ Bias ∣ Tolerance × 100 % = 0.00040 " 0.0040 " × 100 % = 10.0 % \% \text{Bias} = \frac{|\text{Bias}|}{\text{Tolerance}} \times 100\% = \frac{0.00040"}{0.0040"} \times 100\% = \mathbf{10.0\%} % Bias = Tolerance ∣ Bias ∣ ​ × 100% = 0.0040" 0.00040" ​ × 100% = 10.0%

Step 4: Perform One-Sample t t t -Test for Statistical Significance

Degrees of freedom: d f = n − 1 = 15 − 1 = 14 df = n - 1 = 15 - 1 = 14 df = n − 1 = 15 − 1 = 14 .

Standard error of the mean: s X ˉ = s n = 0.0001195 " 15 = 0.0001195 " 3.87298 ≈ 0.00003086 " s_{\bar{X}} = \frac{s}{\sqrt{n}} = \frac{0.0001195"}{\sqrt{15}} = \frac{0.0001195"}{3.87298} \approx 0.00003086" s X ˉ ​ = n ​ s ​ = 15 ​ 0.0001195" ​ = 3.87298 0.0001195" ​ ≈ 0.00003086"

Compute t calc t_{\text{calc}} t calc ​ : t calc = X ˉ − X ref s X ˉ = 0.00040 " 0.00003086 " ≈ 12.96 t_{\text{calc}} = \frac{\bar{X} - X_{\text{ref}}}{s_{\bar{X}}} = \frac{0.00040"}{0.00003086"} \approx 12.96 t calc ​ = s X ˉ ​ X ˉ − X ref ​ ​ = 0.00003086" 0.00040" ​ ≈ 12.96

Look up critical t t t at α = 0.05 \alpha = 0.05 α = 0.05 (two-tailed, d f = 14 df = 14 df = 14 ): t 0.025 , 14 = 2.145 t_{0.025, 14} = 2.145 t 0.025 , 14 ​ = 2.145 .

Step 5: Interpretation and Confidence Interval

Since ∣ t calc ∣ = 12.96 > 2.145 |t_{\text{calc}}| = 12.96 > 2.145 ∣ t calc ​ ∣ = 12.96 > 2.145 , the bias is statistically significant ( p < 0.0001 p < 0.0001 p < 0.0001 ).

95 % 95\% 95% Confidence Interval for Bias: CI = 0.00040 " ± ( 2.145 × 0.00003086 " ) = 0.00040 " ± 0.000066 " = [ + 0.000334 " , + 0.000466 " ] \text{CI} = 0.00040" \pm (2.145 \times 0.00003086") = 0.00040" \pm 0.000066" = [\mathbf{+0.000334", +0.000466"}] CI = 0.00040" ± ( 2.145 × 0.00003086" ) = 0.00040" ± 0.000066" = [ + 0.000334" , + 0.000466" ]

Because the confidence interval does not contain zero, the micrometer exhibits an unadjusted systematic positive offset of + 0.0004 " +0.0004" + 0.0004" . The technician must zero-reset the micrometer barrel before releasing it for production inspection.

Worked Example 2: Linearity Study Regression

Scenario : A linearity study is conducted on an electronic height gage across five calibrated reference standards spanning 1.000 " 1.000" 1.000" to 5.000 " 5.000" 5.000" . The tolerance for parts measured by this gage is 0.0200 " 0.0200" 0.0200" . The average bias values obtained across 10 repeated trials per part are:

Part 1 ( X 1 = 1.000 " X_1 = 1.000" X 1 ​ = 1.000" ): B ˉ 1 = + 0.0002 " \bar{B}_1 = +0.0002" B ˉ 1 ​ = + 0.0002"

Part 2 ( X 2 = 2.000 " X_2 = 2.000" X 2 ​ = 2.000" ): B ˉ 2 = + 0.0005 " \bar{B}_2 = +0.0005" B ˉ 2 ​ = + 0.0005"

Part 3 ( X 3 = 3.000 " X_3 = 3.000" X 3 ​ = 3.000" ): B ˉ 3 = + 0.0008 " \bar{B}_3 = +0.0008" B ˉ 3 ​ = + 0.0008"

Part 4 ( X 4 = 4.000 " X_4 = 4.000" X 4 ​ = 4.000" ): B ˉ 4 = + 0.0011 " \bar{B}_4 = +0.0011" B ˉ 4 ​ = + 0.0011"

Part 5 ( X 5 = 5.000 " X_5 = 5.000" X 5 ​ = 5.000" ): B ˉ 5 = + 0.0014 " \bar{B}_5 = +0.0014" B ˉ 5 ​ = + 0.0014"

Step 1: Calculate Summary Statistics X ˉ = 1 + 2 + 3 + 4 + 5 5 = 3.000 " \bar{X} = \frac{1 + 2 + 3 + 4 + 5}{5} = 3.000" X ˉ = 5 1 + 2 + 3 + 4 + 5 ​ = 3.000" B ˉ = 0.0002 + 0.0005 + 0.0008 + 0.0011 + 0.0014 5 = 0.00080 " \bar{B} = \frac{0.0002 + 0.0005 + 0.0008 + 0.0011 + 0.0014}{5} = 0.00080" B ˉ = 5 0.0002 + 0.0005 + 0.0008 + 0.0011 + 0.0014 ​ = 0.00080"

Step 2: Compute Regression Slope ( b b b )

Part i i i X i − X ˉ X_i - \bar{X} X i ​ − X ˉ ( X i − X ˉ ) 2 (X_i - \bar{X})^2 ( X i ​ − X ˉ ) 2 B i − B ˉ B_i - \bar{B} B i ​ − B ˉ ( X i − X ˉ ) ( B i − B ˉ ) (X_i - \bar{X})(B_i - \bar{B}) ( X i ​ − X ˉ ) ( B i ​ − B ˉ )

  • 1 − 2.0 -2.0 − 2.0 4.0 4.0 4.0 − 0.0006 -0.0006 − 0.0006 + 0.0012 +0.0012 + 0.0012
  • 2 − 1.0 -1.0 − 1.0 1.0 1.0 1.0 − 0.0003 -0.0003 − 0.0003 + 0.0003 +0.0003 + 0.0003
  • 3 0.0 0.0 0.0 0.0 0.0 0.0 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
  • 4 + 1.0 +1.0 + 1.0 1.0 1.0 1.0 + 0.0003 +0.0003 + 0.0003 + 0.0003 +0.0003 + 0.0003
  • 5 + 2.0 +2.0 + 2.0 4.0 4.0 4.0 + 0.0006 +0.0006 + 0.0006 + 0.0012 +0.0012 + 0.0012

Sum 10.0 10.0 10.0 + 0.0030 +0.0030 + 0.0030 b = ∑ ( X i − X ˉ ) ( B i − B ˉ ) ∑ ( X i − X ˉ ) 2 = 0.0030 10.0 = 0.00030 in/in b = \frac{\sum (X_i - \bar{X})(B_i - \bar{B})}{\sum (X_i - \bar{X})^2} = \frac{0.0030}{10.0} = \mathbf{0.00030\text{ in/in}} b = ∑ ( X i ​ − X ˉ ) 2 ∑ ( X i ​ − X ˉ ) ( B i ​ − B ˉ ) ​ = 10.0 0.0030 ​ = 0.00030 in/in

Step 3: Compute Intercept ( a a a ) a = B ˉ − b X ˉ = 0.00080 − ( 0.00030 × 3.000 ) = 0.00080 − 0.00090 = − 0.00010 " a = \bar{B} - b \bar{X} = 0.00080 - (0.00030 \times 3.000) = 0.00080 - 0.00090 = \mathbf{-0.00010"} a = B ˉ − b X ˉ = 0.00080 − ( 0.00030 × 3.000 ) = 0.00080 − 0.00090 = − 0.00010" Regression Equation: Bias = 0.00030 ( X ref ) − 0.00010 \text{Regression Equation: } \text{Bias} = 0.00030(X_{\text{ref}}) - 0.00010 Regression Equation: Bias = 0.00030 ( X ref ​ ) − 0.00010

Step 4: Calculate Percentage Linearity % Linearity = ∣ b ∣ × 100 % = 0.00030 × 100 % = 0.03 % of scale ( or 0.00030 × 0.0200 " 0.0200 " × 100 % = 0.03 % ) ) \% \text{Linearity} = |b| \times 100\% = 0.00030 \times 100\% = \mathbf{0.03\% \text{ of scale}} \quad (\text{or } \frac{0.00030 \times 0.0200"}{0.0200"} \times 100\% = 0.03\%)) % Linearity = ∣ b ∣ × 100% = 0.00030 × 100% = 0.03% of scale ( or 0.0200" 0.00030 × 0.0200" ​ × 100% = 0.03% ))

Metrological Interpretation : The gage exhibits a progressive positive lead error of + 0.00030 " +0.00030" + 0.00030" per inch of travel. Because R 2 = 1.00 R^2 = 1.00 R 2 = 1.00 and the slope is non-zero, this linearity error indicates mechanical pitch wear along the vertical column rack, which cannot be cured by a simple zero adjustment.

  • Technician Inspection Scenarios & Common Exam Traps
  • Real-World Shop Scenario: Bore Gage Calibration Drift

A machining cell operator uses an electronic dial bore gage to inspect cylinder liners. At 7:00 AM, the gage is mastered inside the air-conditioned quality lab ( 68 ∘ F 68^\circ\text{F} 6 8 ∘ F ) using a master ring gage. By 2:00 PM, the shop-floor ambient temperature reaches 92 ∘ F 92^\circ\text{F} 9 2 ∘ F , and the bore gage indicates that cylinder liners are running undersize, prompting the operator to adjust the boring bar tool offset. At 4:00 PM, the quality auditor reinspects the parts in the lab and finds them oversized! Why did this happen?

Root Cause Analysis : The technician mastered the bore gage at 68 ∘ F 68^\circ\text{F} 6 8 ∘ F , but used it at 92 ∘ F 92^\circ\text{F} 9 2 ∘ F . The aluminum gage extension rod expanded significantly in the shop heat, shifting the mechanical zero point (stability drift). Furthermore, the warm cylinder liners expanded during machining and contracted upon cooling in the lab. The technician failed to re-zero the gage on the shop floor and neglected thermal soaking rules.

Common Exam Traps for CQT Candidates

Exam Trap 1: Confusing Bias with Repeatability : Bias is systematic error (distance from target center). Repeatability is random error (cluster width). Adjusting the zero screw fixes bias, but will never fix poor repeatability!

Exam Trap 2: Misinterpreting the t t t -Test Result : If calculated t t t is less than critical t t t ( p > 0.05 p > 0.05 p > 0.05 ), bias is statistically zero (inconsequential random noise). Do not attempt to adjust or calibrate a gage when the bias is statistically insignificant.

Exam Trap 3: Linearity vs. Calibration Offset : If a gage has a constant bias of + 0.001 " +0.001" + 0.001" at 1 " 1" 1" , 2 " 2" 2" , and 3 " 3" 3" , the slope is b = 0 b = 0 b = 0 . This gage has zero linearity error , but high bias. Linearity requires the bias to change across the measurement range.

Exam Trap 4: Stability Sample Type : A stability study must always measure the exact same reference standard or master part. If an inspector measures different production parts over time, the chart reflects part manufacturing variation, not measurement system stability.