Study guide
Outlier Review and Worked Type A Analysis
ASQ Certified Calibration Technician CCT
Track your progress as you read
Finished sections get a check mark.
1 SI Definitions and Realizations
The Seven SI Base Units and the 2019 Revision Realizing SI Units in the Laboratory
2 Derived Units, Prefixes, and Constants
Derived Units and Physical Calculations SI Prefixes and Compound-Unit Conversions Physical Constants and Their Applications
3 Measurement Standards and Selection
Types and Roles of Measurement Standards Selecting Standards for an Operational Hierarchy
4 Traceability and Standard Substitution
Traceability Evidence and Impact Review Evaluating and Authorizing Standard Substitution
5 Dimensional and Mechanical Metrology
Linear Displacement and Dimensional Instruments Geometry, Thermal Expansion, and Mass Force, Torque, Pressure, Density, and Viscosity
6 Temperature, Humidity, Volume, and Flow
Temperature Sensors and Thermometer Corrections Humidity, Volume, and Flow
7 Electrical, Optical, and Time Measurements
Voltage, Current, Resistance, and AC Measurement Time, Frequency, Light, Power, and Energy
8 Measurement Methods
Direct, Indirect, and Ratio Methods Transfer, Differential, and UUT Substitution Methods
9 Instrument Characteristics and Measurement Data
Sensitivity, Variability, and Precision Bias, Linearity, Stability, and Hysteresis Data Formats, Readability, and Suitability Resolution, Confidentiality, and Data Integrity
10 Specifications and Error Sources
Interpreting IM&TE Specification Terms Specification Calculations and Operating Qualifiers Instrument, Operator, and Circuit Errors Environmental, Geometric, and Interference Errors
11 Measurement Assurance
Check Standards and Statistical Process Control Proficiency Comparisons and Gage R&R
12 Procedures and Governing Requirements
Procedure Structure and Published Practice Sources Method Development, Deviations, and Handling Regulations, Contracts, and Requirement Precedence
13 Instrument Adjustment and Standardization
Calibration, Zero, and Span Adjustment Linearization, Nulling, and Digital Adjustment
14 Calibration Environment
Temperature and Humidity Control ESD, Dust, Vibration, and Pressure
15 Operations, Roles, and Scheduling
Receiving, As-Found Testing, and Calibration Flow Asset Identification, Tracking, and Release Laboratory Roles and Initial Scheduling Interval Review, Overdue Control, and Workloads
16 Method and Software Validation
Method Verification and Validation Studies Software, Firmware, and Spreadsheet Validation
17 Records and Formal Reporting
Controlled Records and Calibration Certificates Status Labels and Out-of-Tolerance Reporting
18 Uncertainty Concepts and Decision Risk
Error, Uncertainty, and Measurand Definition TAR, TUR, Tolerance Consumption, and Capability Conformity Decisions and False-Accept Risk Guard Bands and Conformity Reporting
19 Uncertainty Models and Budgets
Measurement Models and Uncertainty Contributions Distribution Selection and Budget Assembly
20 Type A and Type B Evaluation
Type A Statistics and Repeatability Outlier Review and Worked Type A Analysis Type B Distribution Models
21 Uncertainty Reporting and Applied Mathematics
Propagation, Covariance, and Effective Degrees of Freedom Coverage Factors and Expanded-Uncertainty Reporting Notation, Unit Conversions, Ratios, and Decibels Rounding, Algebra, Graphs, and Angular Conversions
22 Quality Systems and Tools
Quality-System Components, Planning, and Improvement The Seven Basic Quality Tools
23 Audits and Corrective Action
Audit Types, Planning, and Nonconformance Control Impact Assessment, Root Cause, and Effectiveness
24 Safety, Standards, and Professional Conduct
Electrical, Pressure, Chemical, and Laboratory Hazards PPE, Safety Data Sheets, 6S, and Safe Restoration Quality Standards, GUM, and VIM Accreditation Scope and Professional Ethics
Outlier Review and Worked Type A Analysis
Key Takeaways Explain This Topic AI
A Grubbs test assumes an appropriate approximately normal model and does not alone authorize data deletion.
Preserve original observations and document the evidence and method supporting any exclusion or repeat run.
Use the spread of individual observations or the mean according to the actual reported measurement model.
Last updated: October 2026
Outlier Detection and Metrological Data Rejection Rules
During a series of repeated measurements, an occasional observation may appear widely disparate from the remaining data. Metrologists use formal statistical tests to determine whether an anomalous point is statistically significant.
Grubbs' Test
Grubbs' test (also known as the maximum normed residual test) is a commonly used method for detecting a single outlier in a sample that approximately follows a normal distribution.
Test Procedure:
Step 1. Calculate the sample mean x ˉ \bar{x} x ˉ and sample standard deviation s s s including the suspected outlier. Step 2. Compute the Grubbs test statistic G G G : G = max i = 1 , … , n ∣ x i − x ˉ ∣ s = ∣ x suspect − x ˉ ∣ s G = \frac{\max_{i=1,\dots,n} |x_i - \bar{x}|}{s} = \frac{|x_{\text{suspect}} - \bar{x}|}{s} G = s max i = 1 , … , n ∣ x i − x ˉ ∣ = s ∣ x suspect − x ˉ ∣
Step 3. Compare calculated G G G with the critical value G crit G_{\text{crit}} G crit from the Grubbs table for sample size n n n at significance level α = 0.05 \alpha = 0.05 α = 0.05 (or α = 0.01 \alpha = 0.01 α = 0.01 ):
If G > G crit G > G_{\text{crit}} G > G crit , the suspected reading is statistically classified as an outlier at the selected significance level under the test assumptions.
Sample Size ( n n n ) Critical Value G crit G_{\text{crit}} G crit ( α = 0.05 \alpha = 0.05 α = 0.05 ) Critical Value G crit G_{\text{crit}} G crit ( α = 0.01 \alpha = 0.01 α = 0.01 )
- 3 1.1543 1.1547
- 4 1.4812 1.4962
- 5 1.7150 1.7637
- 6 1.8871 1.9728
- 7 2.0200 2.1391
- 8 2.1266 2.2744
- 9 2.2150 2.3868
- 10 2.2900 2.4821
The table above is for the two-sided maximum-absolute-residual test. A preselected upper- or lower-tail test uses different critical values. Do not mix a one-sided table with the two-sided statistic.
Dixon's Q Test
Dixon's Q test is a quick ranking method used for small sample sizes ( 3 ≤ n ≤ 10 3 \le n \le 10 3 ≤ n ≤ 10 ):
Step 1. Arrange the data in ascending order: x 1 ≤ x 2 ≤ ⋯ ≤ x n x_1 \le x_2 \le \dots \le x_n x 1 ≤ x 2 ≤ ⋯ ≤ x n . Step 2. Identify the suspect value (either x 1 x_1 x 1 or x n x_n x n ) and compute the experimental ratio Q calc Q_{\text{calc}} Q calc : Q calc = ∣ x suspect − x nearest neighbor ∣ x max − x min = Gap Range Q_{\text{calc}} = \frac{|x_{\text{suspect}} - x_{\text{nearest neighbor}}|}{x_{\max} - x_{\min}} = \frac{\text{Gap}}{\text{Range}} Q calc = x m a x − x m i n ∣ x suspect − x nearest neighbor ∣ = Range Gap
Step 3. Compare Q calc Q_{\text{calc}} Q calc against tabulated Q crit Q_{\text{crit}} Q crit . If Q calc > Q crit Q_{\text{calc}} > Q_{\text{crit}} Q calc > Q crit , the reading is statistically anomalous.
ISO/IEC 17025 Compliance: Permitted vs. Prohibited Data Rejection
Caution
The Strict Metrological Rule on Data Rejection: A statistical test (Grubbs or Dixon) never gives a calibration technician the authority to discard a data point on its own! Statistical tests only establish that a data point is improbable under the assumed normal distribution.
Under ISO/IEC 17025 accreditation requirements and good metrological practice:
When Data Rejection Is Prohibited:
A technician cannot delete a reading simply because it "ruins the uncertainty budget" or causes the test uncertainty ratio (TUR) to fall below 4 : 1 4:1 4 : 1 .
A technician cannot silently discard a reading merely because a statistic flags it; investigate the evidence and follow a justified, documented method.
Undocumented "cherry-picking" to manufacture a favorable calibration outcome is data falsification.
When Data Rejection Is Permitted:
A data point may be excluded under a justified documented procedure, for example when an assignable physical cause is verified and recorded on the calibration worksheet.
Permissible assignable causes include: an observed electrical line voltage transient, an acoustic disturbance or floor vibration during an optical measurement, a confirmed speck of dust or lint on a gage block face, operator error in recording a digit, or an unseated connector.
Remedial Action: Preserve original observations and investigate flagged points. If evidence establishes an invalid observation, document the exclusion and follow the approved replacement or repeat-run method. Statistical procedures can support documented treatment under their assumptions; a flag alone is not permission to discard an inconvenient result.
Step-by-Step Worked Calibration Example: Type A Analysis
A calibration technician calibrates a precision digital voltmeter by applying a nominal 10.00000 V 10.00000\text{ V} 10.00000 V DC reference standard from a Josephson-traceable calibrator. The technician records n = 10 n = 10 n = 10 repeated voltage indications under repeatability conditions: x 1 = 10.00012 V x 2 = 10.00015 V x 3 = 10.00009 V x 4 = 10.00014 V x 5 = 10.00011 V x 6 = 10.00016 V x 7 = 10.00013 V x 8 = 10.00010 V x 9 = 10.00014 V x 10 = 10.00026 V \begin{matrix}
x_1 = 10.00012\text{ V} & x_2 = 10.00015\text{ V} & x_3 = 10.00009\text{ V} & x_4 = 10.00014\text{ V} & x_5 = 10.00011\text{ V} \\
x_6 = 10.00016\text{ V} & x_7 = 10.00013\text{ V} & x_8 = 10.00010\text{ V} & x_9 = 10.00014\text{ V} & x_{10} = 10.00026\text{ V}
\end{matrix} x 1 = 10.00012 V x 6 = 10.00016 V x 2 = 10.00015 V x 7 = 10.00013 V x 3 = 10.00009 V x 8 = 10.00010 V x 4 = 10.00014 V x 9 = 10.00014 V x 5 = 10.00011 V x 10 = 10.00026 V
Notice reading x 10 = 10.00026 V x_{10} = 10.00026\text{ V} x 10 = 10.00026 V appears unusually high. Let us perform the complete Type A analysis and outlier investigation.
Step 1: Compute Sample Mean ( x ˉ \bar{x} x ˉ ) ∑ i = 1 10 x i = 100.00140 V \sum_{i=1}^{10} x_i = 100.00140\text{ V} i = 1 ∑ 10 x i = 100.00140 V x ˉ = 100.00140 V 10 = 10.000140 V \bar{x} = \frac{100.00140\text{ V}}{10} = 10.000140\text{ V} x ˉ = 10 100.00140 V = 10.000140 V
Step 2: Compute Residuals and Sum of Squared Deviations
- Observation Reading (V) Residual (µV) Squared residual (µV²)
- 1 10.00012 -20 400
- 2 10.00015 10 100
- 3 10.00009 -50 2500
- 4 10.00014 0 0
- 5 10.00011 -30 900
- 6 10.00016 20 400
- 7 10.00013 -10 100
- 8 10.00010 -40 1600
- 9 10.00014 0 0
- 10 10.00026 120 14400
- Sum 100.00140 0 20400
The squared-residual sum is 20400 ( μ V ) 2 = 2.040 × 10 − 8 V 2 20400\ (\mu\mathrm{V})^2=2.040\times10^{-8}\ \mathrm{V}^2 20400 ( μ V ) 2 = 2.040 × 1 0 − 8 V 2 .
Step 3: Compute Sample Variance ( s 2 s^2 s 2 ) and Sample Standard Deviation ( s s s ) ν = n − 1 = 10 − 1 = 9 \nu = n - 1 = 10 - 1 = 9 ν = n − 1 = 10 − 1 = 9 s 2 = ∑ ( x i − x ˉ ) 2 n − 1 = 2.040 × 10 − 8 V 2 9 = 2.2667 × 10 − 9 V 2 s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1} = \frac{2.040 \times 10^{-8}\text{ V}^2}{9} = 2.2667 \times 10^{-9}\text{ V}^2 s 2 = n − 1 ∑ ( x i − x ˉ ) 2 = 9 2.040 × 1 0 − 8 V 2 = 2.2667 × 1 0 − 9 V 2 s = 2.2667 × 10 − 9 V 2 = 4.761 × 10 − 5 V = 47.61 μ V s = \sqrt{2.2667 \times 10^{-9}\text{ V}^2} = 4.761 \times 10^{-5}\text{ V} = 47.61\ \mu\text{V} s = 2.2667 × 1 0 − 9 V 2 = 4.761 × 1 0 − 5 V = 47.61 μ V
Step 4: Perform Grubbs' Outlier Test on x 10 x_{10} x 10
Suspect value: x 10 = 10.00026 V x_{10} = 10.00026\text{ V} x 10 = 10.00026 V . Residual magnitude: ∣ 10.00026 − 10.000140 ∣ = 120 μ V |10.00026 - 10.000140| = 120\ \mu\text{V} ∣10.00026 − 10.000140∣ = 120 μ V . G = ∣ x 10 − x ˉ ∣ s = 120 μ V 47.61 μ V = 2.5205 G = \frac{|x_{10} - \bar{x}|}{s} = \frac{120\ \mu\text{V}}{47.61\ \mu\text{V}} = 2.5205 G = s ∣ x 10 − x ˉ ∣ = 47.61 μ V 120 μ V = 2.5205
From the Grubbs critical value table for n = 10 n = 10 n = 10 at α = 0.05 \alpha = 0.05 α = 0.05 : G crit ( 10 , 0.05 ) = 2.2900 G_{\text{crit}}(10, 0.05) = 2.2900 G crit ( 10 , 0.05 ) = 2.2900
Because G = 2.521 > 2.2900 G = 2.521 > 2.2900 G = 2.521 > 2.2900 , observation x 10 x_{10} x 10 is statistically flagged as an outlier at significance level α = 0.05 \alpha=0.05 α = 0.05 under the two-sided test.
Hypothetical investigation: Assume synchronized instrument diagnostics and electrical monitoring establish that interference invalidated observation 10. A coincident HVAC start alone would not prove causation. Preserve original data and follow the approved documented exclusion and replacement or repeat-run method. The nine-reading calculation below illustrates arithmetic after justified exclusion, not permission to trim an inconvenient result.
Step 5: Recalculate Statistics for the Valid Sample ( n = 9 n = 9 n = 9 )
Sum of the remaining 9 readings: ∑ x i = 90.00114 V \sum x_i = 90.00114\text{ V} ∑ x i = 90.00114 V . x ˉ new = 90.00114 V 9 = 10.0001267 V ≈ 10.000127 V \bar{x}_{\text{new}} = \frac{90.00114\text{ V}}{9} = 10.0001267\text{ V} \approx 10.000127\text{ V} x ˉ new = 9 90.00114 V = 10.0001267 V ≈ 10.000127 V ∑ i = 1 9 ( x i − x ˉ new ) 2 = 4.400 × 10 − 9 V 2 \sum_{i=1}^9 (x_i - \bar{x}_{\text{new}})^2 = 4.400 \times 10^{-9}\text{ V}^2 i = 1 ∑ 9 ( x i − x ˉ new ) 2 = 4.400 × 1 0 − 9 V 2 ν new = 9 − 1 = 8 \nu_{\text{new}} = 9 - 1 = 8 ν new = 9 − 1 = 8 s new = 4.400 × 10 − 9 8 = 2.3452 × 10 − 5 V = 23.45 μ V s_{\text{new}} = \sqrt{\frac{4.400 \times 10^{-9}}{8}} = 2.3452 \times 10^{-5}\text{ V} = 23.45\ \mu\text{V} s new = 8 4.400 × 1 0 − 9 = 2.3452 × 1 0 − 5 V = 23.45 μ V
Step 6: Compute Standard Uncertainty of the Mean u ( x ˉ ) u(\bar{x}) u ( x ˉ )
Because the calibration certificate will report the mean of the 9 readings as the calibrated voltage: u ( x ˉ ) = s new n = 23.45 μ V 9 = 23.45 μ V 3 = 7.82 μ V u(\bar{x}) = \frac{s_{\text{new}}}{\sqrt{n}} = \frac{23.45\ \mu\text{V}}{\sqrt{9}} = \frac{23.45\ \mu\text{V}}{3} = 7.82\ \mu\text{V} u ( x ˉ ) = n s new = 9 23.45 μ V = 3 23.45 μ V = 7.82 μ V
Notice the metrological impact:
The sample standard deviation s = 23.45 μ V s = 23.45\ \mu\text{V} s = 23.45 μ V represents the process repeatability (dispersion of single voltmeter readings).
The standard uncertainty of the mean u ( x ˉ ) = 7.82 μ V u(\bar{x}) = 7.82\ \mu\text{V} u ( x ˉ ) = 7.82 μ V is the Type A standard uncertainty that enters the combined uncertainty budget for the reported mean value.
Official references (checked October 10, 2026): NIST Grubbs test definition and critical-value formula .