Study guide
7.1 Measurement System Fundamentals: Precision, Accuracy & Bias
ASQ Certified Six Sigma Yellow Belt CSSYB
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
About the ASQ CSSYB Exam How to Use This Study Guide
2 Chapter 1: Six Sigma & Lean Fundamentals
1.1 Six Sigma Foundations, Principles & History 1.2 Lean Foundations, Principles & The 8 Wastes 1.3 Six Sigma Roles & Responsibilities
3 Chapter 2: Team Basics & Dynamics
2.1 Team Types & Stages of Team Evolution 2.2 Team Decision-Making & Idea Generation Tools 2.3 Communication Methods & Meeting Management
4 Chapter 3: Quality Tools & Six Sigma Metrics
3.1 The 7 Basic Quality Tools 3.2 Six Sigma Process Performance Metrics & COPQ
5 Chapter 4: Define Phase — Project Identification & Scope
4.1 Voice of the Customer (VOC) & Critical-to-Quality (CTQ) 4.2 Project Selection & Stakeholder Analysis 4.3 Process Definition with SIPOC & Supply Chain Basics
6 Chapter 5: Define Phase — Project Management Basics
5.1 The Project Charter & Problem Statements 5.2 Project Planning: WBS, Gantt Charts & Communication Plans 5.3 Project Management Tools & Tollgate Reviews
7 Chapter 6: Measure Phase — Data Collection & Statistics
6.1 Basic Statistics: Central Tendency & Dispersion 6.2 Data Types & Classification 6.3 Data Collection Plans, Operational Definitions & Sampling 6.4 Data Collection Techniques: Check Sheets, Checklists, Surveys & Interviews
8 Chapter 7: Measure Phase — Measurement System Analysis
7.1 Measurement System Fundamentals: Precision, Accuracy & Bias 7.2 Gauge Repeatability & Reproducibility (GR&R)
9 Chapter 8: Analyze Phase — Process & Root Cause Analysis
8.1 Process Analysis: 5S & Value Analysis 8.2 Failure Mode and Effects Analysis (FMEA) 8.3 Root Cause Analysis: 5 Whys, Process Mapping, 8D, Force-Field & Matrix Charts
10 Chapter 9: Analyze Phase — Corrective Action & Variation
9.1 Corrective & Preventive Action Processes 9.2 Common Cause vs Special Cause Variation 9.3 Basic Distribution Types: Normal, Binomial & Poisson
11 Chapter 10: Analyze Phase — Relationships & Hypothesis Testing
10.1 Correlation & Scatter Plots 10.2 Simple Linear Regression & Prediction 10.3 Hypothesis Testing Concepts: Hypotheses, Errors & P-Values
12 Chapter 11: Improve Phase — Implementation Techniques
11.1 Kaizen & Kaizen Blitz Events 11.2 Plan-Do-Check-Act (PDCA) & Solution Selection 11.3 Cost-Benefit Analysis & Financial Validation
13 Chapter 12: Control Phase — Sustaining Improvements
12.1 Control Plans & Process Monitoring 12.2 Statistical Process Control: X-bar & R Charts 12.3 Document Control, Standard Operating Procedures & Project Hand-off
7.1 Measurement System Fundamentals: Precision, Accuracy & Bias
Key Takeaways Explain This Topic AI
Total observed process variance is the mathematical sum of true process variance and measurement system error variance: σ²total = σ²process + σ²measurement.
Accuracy reflects the closeness of agreement between the average of observed measurements and a certified reference master value (systematic error), whereas precision reflects the closeness of repeated measurements to each other (random error).
Bias is the quantitative difference between the observed measurement average and an accepted reference master standard.
Linearity evaluates the consistency of measurement bias across the entire operating range of the gauge, while stability (drift) evaluates bias consistency over extended calendar time.
The 10-to-1 Resolution Rule requires an instrument's smallest readable increment to be no larger than one-tenth (10%) of the tolerance width or of the six-sigma process spread.
Last updated: September 2026
Measurement System Fundamentals: Precision, Accuracy & Bias
Quick Answer: Measurement System Analysis (MSA) evaluates data collection tools, fixtures, and operators to ensure observations reflect true process behavior rather than measurement noise. In the Six Sigma Measure phase, total observed variation is partitioned into process variation and measurement error ( σ t o t a l 2 = σ p r o c e s s 2 + σ m e a s u r e m e n t 2 \sigma^2_{total} = \sigma^2_{process} + \sigma^2_{measurement} σ t o t a l 2 = σ p rocess 2 + σ m e a s u re m e n t 2 ). Metrology assesses accuracy (systematic error: bias, linearity, and stability) and precision (random error: repeatability and reproducibility). Under the 10-to-1 Resolution Rule, an instrument's smallest readable graduation must not exceed 10% of the specification tolerance or process spread. Independent CSSYB study guide by ExamFinch.
Purpose of MSA in the Measure Phase
In Lean Six Sigma, data informs every root-cause hypothesis and capability calculation. However, raw process data never reflects pure process performance alone. Every observation combines true part-to-part variation with error introduced by instruments, fixtures, environment, and human handling.
Total observed variance ( σ t o t a l 2 \sigma^2_{total} σ t o t a l 2 ) is the mathematical sum of true process variance ( σ p r o c e s s 2 \sigma^2_{process} σ p rocess 2 ) and measurement system variance ( σ m e a s u r e m e n t 2 \sigma^2_{measurement} σ m e a s u re m e n t 2 ): σ t o t a l 2 = σ p r o c e s s 2 + σ m e a s u r e m e n t 2 \sigma^2_{total} = \sigma^2_{process} + \sigma^2_{measurement} σ t o t a l 2 = σ p rocess 2 + σ m e a s u re m e n t 2
Validating the measurement system is a mandatory gate in the DMAIC Measure phase. If measurement noise is excessive, two operational risks arise:
Producer's Risk (Type I Error / False Alarm): Conforming product is mistakenly rejected because measurement noise pushes recorded values outside specification limits, causing wasteful scrap and unwarranted machine adjustments.
Consumer's Risk (Type II Error / Missed Defect): Nonconforming product is mistakenly accepted and released because measurement error pulls out-of-spec dimensions inside tolerance boundaries.
A capable measurement system ensures that observed variability reflects actual process realities rather than testing noise.
Precision vs. Accuracy: The Metrological Distinction
Precision and accuracy represent fundamentally different types of measurement error that must be evaluated separately:
Accuracy (Location / Systematic Error): The closeness of agreement between the average of observed measurements and an accepted, certified reference master value. Accuracy errors are systematic offsets caused by uncalibrated zero points, physical wear, or uncorrected thermal expansion.
Precision (Spread / Random Error): The closeness of agreement among repeated measurements on the same part under identical conditions. Precision reflects random variability caused by mechanical play, electrical noise, or minor operator seating differences, completely independent of the true reference value.
Dimension Metrological Definition Error Type Primary Sources Corrective Actions
Accuracy Closeness of measurement average to certified standard Systematic Zero-point drift, wear, thermal expansion Recalibration against traceable standards, zero resets
Precision Closeness of repeated measurements to each other Random Mechanical play, electrical noise, seating variations Fixture stiffening, equipment maintenance, SOP training
Core MSA Properties: Bias, Linearity, and Stability
Measurement system accuracy is evaluated across three primary dimensions:
1. Bias (Accuracy at a Single Point)
Bias is the quantitative difference between the observed average of measurements and a certified reference master value: Bias = X ˉ o b s e r v e d − X r e f e r e n c e \text{Bias} = \bar{X}_{observed} - X_{reference} Bias = X ˉ o b ser v e d − X re f ere n ce
To assess bias, a certified master standard (such as a calibrated gauge block) is measured 10 to 15 times under production conditions. A one-sample t t t -test determines if the sample mean ( X ˉ \bar{X} X ˉ ) differs significantly from the reference value ( X r e f e r e n c e X_{reference} X re f ere n ce ) at α = 0.05 \alpha = 0.05 α = 0.05 . Significant bias requires physical calibration or digital zero-offset adjustment.
2. Linearity (Accuracy Across the Operating Range)
Linearity evaluates whether bias remains constant across the instrument's entire operating range. A gauge may have zero bias when measuring small components but develop substantial bias when measuring larger parts.
To assess linearity, practitioners measure 3 to 5 reference masters covering 10% to 90% of the gauge's scale. Observed bias is plotted against reference values, and a linear regression line is fitted: Bias = b 0 + b 1 × ( Reference Value ) \text{Bias} = b_0 + b_1 \times (\text{Reference Value}) Bias = b 0 + b 1 × ( Reference Value )
If the regression slope ( b 1 b_1 b 1 ) is statistically indistinguishable from zero, linearity is acceptable. A significant slope indicates that the instrument requires multi-point span recalibration or software correction.
3. Stability / Drift (Accuracy Over Extended Time)
Stability represents the total variation in measurements obtained with a measurement system on the same master part over an extended timeframe.
Over weeks or months, gauge accuracy can degrade due to mechanical wear, sensor aging, or ambient temperature cycling. Stability is monitored by measuring a reference master at scheduled intervals (such as daily or once per shift) and plotting the results on an Individuals and Moving Range ( I − M R I-MR I − MR ) or X ˉ − R \bar{X}-R X ˉ − R control chart. Any out-of-control signals indicate that the system has drifted and requires maintenance or recalibration.
Resolution and Discrimination: The 10-to-1 Rule
Resolution (also called discrimination or readability) is the smallest detectable unit or graduation that a measuring device can distinguish and display.
When an instrument lacks adequate resolution, observations cluster into coarse, identical numerical steps. This discretization masks true variation, produces stepped histograms, and inflates standard deviation calculations.
The 10-to-1 Rule (Rule of Ten / Gage Maker's Rule)
To ensure sufficient discrimination, the instrument's smallest increment of measurement must be at most one-tenth (10%) of the process tolerance width (USL - LSL) or one-tenth of six-sigma process variation ( 6 σ p r o c e s s 6\sigma_{process} 6 σ p rocess ): Maximum Allowable Resolution ≤ Tolerance Width 10 = USL − LSL 10 \text{Maximum Allowable Resolution} \le \frac{\text{Tolerance Width}}{10} = \frac{\text{USL} - \text{LSL}}{10} Maximum Allowable Resolution ≤ 10 Tolerance Width = 10 USL − LSL
For example, if a machined pin has an engineering tolerance of 20.00 ± 0.05 mm 20.00 \pm 0.05\text{ mm} 20.00 ± 0.05 mm (tolerance width = 0.10 mm = 0.10\text{ mm} = 0.10 mm ), the gauge resolution must be no coarser than 0.10 / 10 = 0.01 mm 0.10 / 10 = 0.01\text{ mm} 0.10/10 = 0.01 mm (or finer, such as 0.001 mm 0.001\text{ mm} 0.001 mm ). A tool with 0.05 mm 0.05\text{ mm} 0.05 mm resolution fails the 10-to-1 Rule and cannot reliably track process performance.
Calibration Systems and Metrological Traceability
Calibration is the formal comparison of an instrument against an uncompromised standard of known accuracy to detect, report, and eliminate systematic error.
Six Sigma programs require metrological traceability —an unbroken, documented chain of calibrations linking shop-floor tools back to primary standards maintained by organizations such as the National Institute of Standards and Technology (NIST) or ISO/IEC 17025 accredited metrology laboratories.
Core calibration requirements include:
Scheduled Intervals: Periodic checks based on usage frequency and historical drift rates.
Calibration Stickers: Physical labels displaying serial number, calibration date, expiration date, and technician initials.
Environmental Controls: Standard metrology room conditions ( 20 ∘ C ± 1 ∘ C 20^\circ\text{C} \pm 1^\circ\text{C} 2 0 ∘ C ± 1 ∘ C or 68 ∘ F 68^\circ\text{F} 6 8 ∘ F ) to eliminate thermal expansion distortions.
Quarantine Procedures: Immediate isolation of any gauge that fails calibration, followed by retrospective risk evaluation of production lots measured since the last valid check.