Study guide

Outlier Review and Worked Type A Analysis

ASQ Certified Calibration Technician CCT

Track your progress as you read

Finished sections get a check mark.

1 SI Definitions and Realizations

The Seven SI Base Units and the 2019 Revision Realizing SI Units in the Laboratory

2 Derived Units, Prefixes, and Constants

Derived Units and Physical Calculations SI Prefixes and Compound-Unit Conversions Physical Constants and Their Applications

3 Measurement Standards and Selection

Types and Roles of Measurement Standards Selecting Standards for an Operational Hierarchy

4 Traceability and Standard Substitution

Traceability Evidence and Impact Review Evaluating and Authorizing Standard Substitution

5 Dimensional and Mechanical Metrology

Linear Displacement and Dimensional Instruments Geometry, Thermal Expansion, and Mass Force, Torque, Pressure, Density, and Viscosity

6 Temperature, Humidity, Volume, and Flow

Temperature Sensors and Thermometer Corrections Humidity, Volume, and Flow

7 Electrical, Optical, and Time Measurements

Voltage, Current, Resistance, and AC Measurement Time, Frequency, Light, Power, and Energy

8 Measurement Methods

Direct, Indirect, and Ratio Methods Transfer, Differential, and UUT Substitution Methods

9 Instrument Characteristics and Measurement Data

Sensitivity, Variability, and Precision Bias, Linearity, Stability, and Hysteresis Data Formats, Readability, and Suitability Resolution, Confidentiality, and Data Integrity

10 Specifications and Error Sources

Interpreting IM&TE Specification Terms Specification Calculations and Operating Qualifiers Instrument, Operator, and Circuit Errors Environmental, Geometric, and Interference Errors

11 Measurement Assurance

Check Standards and Statistical Process Control Proficiency Comparisons and Gage R&R

12 Procedures and Governing Requirements

Procedure Structure and Published Practice Sources Method Development, Deviations, and Handling Regulations, Contracts, and Requirement Precedence

13 Instrument Adjustment and Standardization

Calibration, Zero, and Span Adjustment Linearization, Nulling, and Digital Adjustment

14 Calibration Environment

Temperature and Humidity Control ESD, Dust, Vibration, and Pressure

15 Operations, Roles, and Scheduling

Receiving, As-Found Testing, and Calibration Flow Asset Identification, Tracking, and Release Laboratory Roles and Initial Scheduling Interval Review, Overdue Control, and Workloads

16 Method and Software Validation

Method Verification and Validation Studies Software, Firmware, and Spreadsheet Validation

17 Records and Formal Reporting

Controlled Records and Calibration Certificates Status Labels and Out-of-Tolerance Reporting

18 Uncertainty Concepts and Decision Risk

Error, Uncertainty, and Measurand Definition TAR, TUR, Tolerance Consumption, and Capability Conformity Decisions and False-Accept Risk Guard Bands and Conformity Reporting

19 Uncertainty Models and Budgets

Measurement Models and Uncertainty Contributions Distribution Selection and Budget Assembly

20 Type A and Type B Evaluation

Type A Statistics and Repeatability Outlier Review and Worked Type A Analysis Type B Distribution Models

21 Uncertainty Reporting and Applied Mathematics

Propagation, Covariance, and Effective Degrees of Freedom Coverage Factors and Expanded-Uncertainty Reporting Notation, Unit Conversions, Ratios, and Decibels Rounding, Algebra, Graphs, and Angular Conversions

22 Quality Systems and Tools

Quality-System Components, Planning, and Improvement The Seven Basic Quality Tools

23 Audits and Corrective Action

Audit Types, Planning, and Nonconformance Control Impact Assessment, Root Cause, and Effectiveness

24 Safety, Standards, and Professional Conduct

Electrical, Pressure, Chemical, and Laboratory Hazards PPE, Safety Data Sheets, 6S, and Safe Restoration Quality Standards, GUM, and VIM Accreditation Scope and Professional Ethics

Outlier Review and Worked Type A Analysis

Key Takeaways Explain This Topic AI

A Grubbs test assumes an appropriate approximately normal model and does not alone authorize data deletion.

Preserve original observations and document the evidence and method supporting any exclusion or repeat run.

Use the spread of individual observations or the mean according to the actual reported measurement model.

Last updated: October 2026

Outlier Detection and Metrological Data Rejection Rules

During a series of repeated measurements, an occasional observation may appear widely disparate from the remaining data. Metrologists use formal statistical tests to determine whether an anomalous point is statistically significant.

Grubbs' Test

Grubbs' test (also known as the maximum normed residual test) is a commonly used method for detecting a single outlier in a sample that approximately follows a normal distribution.

Test Procedure:

Step 1. Calculate the sample mean x ˉ \bar{x} x ˉ and sample standard deviation s s s including the suspected outlier. Step 2. Compute the Grubbs test statistic G G G : G = max ⁡ i = 1 , … , n ∣ x i − x ˉ ∣ s = ∣ x suspect − x ˉ ∣ s G = \frac{\max_{i=1,\dots,n} |x_i - \bar{x}|}{s} = \frac{|x_{\text{suspect}} - \bar{x}|}{s} G = s max i = 1 , … , n ​ ∣ x i ​ − x ˉ ∣ ​ = s ∣ x suspect ​ − x ˉ ∣ ​

Step 3. Compare calculated G G G with the critical value G crit G_{\text{crit}} G crit ​ from the Grubbs table for sample size n n n at significance level α = 0.05 \alpha = 0.05 α = 0.05 (or α = 0.01 \alpha = 0.01 α = 0.01 ):

If G > G crit G > G_{\text{crit}} G > G crit ​ , the suspected reading is statistically classified as an outlier at the selected significance level under the test assumptions.

Sample Size ( n n n ) Critical Value G crit G_{\text{crit}} G crit ​ ( α = 0.05 \alpha = 0.05 α = 0.05 ) Critical Value G crit G_{\text{crit}} G crit ​ ( α = 0.01 \alpha = 0.01 α = 0.01 )

  • 3 1.1543 1.1547
  • 4 1.4812 1.4962
  • 5 1.7150 1.7637
  • 6 1.8871 1.9728
  • 7 2.0200 2.1391
  • 8 2.1266 2.2744
  • 9 2.2150 2.3868
  • 10 2.2900 2.4821

The table above is for the two-sided maximum-absolute-residual test. A preselected upper- or lower-tail test uses different critical values. Do not mix a one-sided table with the two-sided statistic.

Dixon's Q Test

Dixon's Q test is a quick ranking method used for small sample sizes ( 3 ≤ n ≤ 10 3 \le n \le 10 3 ≤ n ≤ 10 ):

Step 1. Arrange the data in ascending order: x 1 ≤ x 2 ≤ ⋯ ≤ x n x_1 \le x_2 \le \dots \le x_n x 1 ​ ≤ x 2 ​ ≤ ⋯ ≤ x n ​ . Step 2. Identify the suspect value (either x 1 x_1 x 1 ​ or x n x_n x n ​ ) and compute the experimental ratio Q calc Q_{\text{calc}} Q calc ​ : Q calc = ∣ x suspect − x nearest neighbor ∣ x max ⁡ − x min ⁡ = Gap Range Q_{\text{calc}} = \frac{|x_{\text{suspect}} - x_{\text{nearest neighbor}}|}{x_{\max} - x_{\min}} = \frac{\text{Gap}}{\text{Range}} Q calc ​ = x m a x ​ − x m i n ​ ∣ x suspect ​ − x nearest neighbor ​ ∣ ​ = Range Gap ​

Step 3. Compare Q calc Q_{\text{calc}} Q calc ​ against tabulated Q crit Q_{\text{crit}} Q crit ​ . If Q calc > Q crit Q_{\text{calc}} > Q_{\text{crit}} Q calc ​ > Q crit ​ , the reading is statistically anomalous.

ISO/IEC 17025 Compliance: Permitted vs. Prohibited Data Rejection

Caution

The Strict Metrological Rule on Data Rejection: A statistical test (Grubbs or Dixon) never gives a calibration technician the authority to discard a data point on its own! Statistical tests only establish that a data point is improbable under the assumed normal distribution.

Under ISO/IEC 17025 accreditation requirements and good metrological practice:

When Data Rejection Is Prohibited:

A technician cannot delete a reading simply because it "ruins the uncertainty budget" or causes the test uncertainty ratio (TUR) to fall below 4 : 1 4:1 4 : 1 .

A technician cannot silently discard a reading merely because a statistic flags it; investigate the evidence and follow a justified, documented method.

Undocumented "cherry-picking" to manufacture a favorable calibration outcome is data falsification.

When Data Rejection Is Permitted:

A data point may be excluded under a justified documented procedure, for example when an assignable physical cause is verified and recorded on the calibration worksheet.

Permissible assignable causes include: an observed electrical line voltage transient, an acoustic disturbance or floor vibration during an optical measurement, a confirmed speck of dust or lint on a gage block face, operator error in recording a digit, or an unseated connector.

Remedial Action: Preserve original observations and investigate flagged points. If evidence establishes an invalid observation, document the exclusion and follow the approved replacement or repeat-run method. Statistical procedures can support documented treatment under their assumptions; a flag alone is not permission to discard an inconvenient result.

Step-by-Step Worked Calibration Example: Type A Analysis

A calibration technician calibrates a precision digital voltmeter by applying a nominal 10.00000 V 10.00000\text{ V} 10.00000 V DC reference standard from a Josephson-traceable calibrator. The technician records n = 10 n = 10 n = 10 repeated voltage indications under repeatability conditions: x 1 = 10.00012 V x 2 = 10.00015 V x 3 = 10.00009 V x 4 = 10.00014 V x 5 = 10.00011 V x 6 = 10.00016 V x 7 = 10.00013 V x 8 = 10.00010 V x 9 = 10.00014 V x 10 = 10.00026 V \begin{matrix}

x_1 = 10.00012\text{ V} & x_2 = 10.00015\text{ V} & x_3 = 10.00009\text{ V} & x_4 = 10.00014\text{ V} & x_5 = 10.00011\text{ V} \\

x_6 = 10.00016\text{ V} & x_7 = 10.00013\text{ V} & x_8 = 10.00010\text{ V} & x_9 = 10.00014\text{ V} & x_{10} = 10.00026\text{ V}

\end{matrix} x 1 ​ = 10.00012 V x 6 ​ = 10.00016 V ​ x 2 ​ = 10.00015 V x 7 ​ = 10.00013 V ​ x 3 ​ = 10.00009 V x 8 ​ = 10.00010 V ​ x 4 ​ = 10.00014 V x 9 ​ = 10.00014 V ​ x 5 ​ = 10.00011 V x 10 ​ = 10.00026 V ​

Notice reading x 10 = 10.00026 V x_{10} = 10.00026\text{ V} x 10 ​ = 10.00026 V appears unusually high. Let us perform the complete Type A analysis and outlier investigation.

Step 1: Compute Sample Mean ( x ˉ \bar{x} x ˉ ) ∑ i = 1 10 x i = 100.00140 V \sum_{i=1}^{10} x_i = 100.00140\text{ V} i = 1 ∑ 10 ​ x i ​ = 100.00140 V x ˉ = 100.00140 V 10 = 10.000140 V \bar{x} = \frac{100.00140\text{ V}}{10} = 10.000140\text{ V} x ˉ = 10 100.00140 V ​ = 10.000140 V

Step 2: Compute Residuals and Sum of Squared Deviations

  • Observation Reading (V) Residual (µV) Squared residual (µV²)
  • 1 10.00012 -20 400
  • 2 10.00015 10 100
  • 3 10.00009 -50 2500
  • 4 10.00014 0 0
  • 5 10.00011 -30 900
  • 6 10.00016 20 400
  • 7 10.00013 -10 100
  • 8 10.00010 -40 1600
  • 9 10.00014 0 0
  • 10 10.00026 120 14400
  • Sum 100.00140 0 20400

The squared-residual sum is 20400 ( μ V ) 2 = 2.040 × 10 − 8 V 2 20400\ (\mu\mathrm{V})^2=2.040\times10^{-8}\ \mathrm{V}^2 20400 ( μ V ) 2 = 2.040 × 1 0 − 8 V 2 .

Step 3: Compute Sample Variance ( s 2 s^2 s 2 ) and Sample Standard Deviation ( s s s ) ν = n − 1 = 10 − 1 = 9 \nu = n - 1 = 10 - 1 = 9 ν = n − 1 = 10 − 1 = 9 s 2 = ∑ ( x i − x ˉ ) 2 n − 1 = 2.040 × 10 − 8 V 2 9 = 2.2667 × 10 − 9 V 2 s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1} = \frac{2.040 \times 10^{-8}\text{ V}^2}{9} = 2.2667 \times 10^{-9}\text{ V}^2 s 2 = n − 1 ∑ ( x i ​ − x ˉ ) 2 ​ = 9 2.040 × 1 0 − 8 V 2 ​ = 2.2667 × 1 0 − 9 V 2 s = 2.2667 × 10 − 9 V 2 = 4.761 × 10 − 5 V = 47.61 μ V s = \sqrt{2.2667 \times 10^{-9}\text{ V}^2} = 4.761 \times 10^{-5}\text{ V} = 47.61\ \mu\text{V} s = 2.2667 × 1 0 − 9 V 2 ​ = 4.761 × 1 0 − 5 V = 47.61 μ V

Step 4: Perform Grubbs' Outlier Test on x 10 x_{10} x 10 ​

Suspect value: x 10 = 10.00026 V x_{10} = 10.00026\text{ V} x 10 ​ = 10.00026 V . Residual magnitude: ∣ 10.00026 − 10.000140 ∣ = 120 μ V |10.00026 - 10.000140| = 120\ \mu\text{V} ∣10.00026 − 10.000140∣ = 120 μ V . G = ∣ x 10 − x ˉ ∣ s = 120 μ V 47.61 μ V = 2.5205 G = \frac{|x_{10} - \bar{x}|}{s} = \frac{120\ \mu\text{V}}{47.61\ \mu\text{V}} = 2.5205 G = s ∣ x 10 ​ − x ˉ ∣ ​ = 47.61 μ V 120 μ V ​ = 2.5205

From the Grubbs critical value table for n = 10 n = 10 n = 10 at α = 0.05 \alpha = 0.05 α = 0.05 : G crit ( 10 , 0.05 ) = 2.2900 G_{\text{crit}}(10, 0.05) = 2.2900 G crit ​ ( 10 , 0.05 ) = 2.2900

Because G = 2.521 > 2.2900 G = 2.521 > 2.2900 G = 2.521 > 2.2900 , observation x 10 x_{10} x 10 ​ is statistically flagged as an outlier at significance level α = 0.05 \alpha=0.05 α = 0.05 under the two-sided test.

Hypothetical investigation: Assume synchronized instrument diagnostics and electrical monitoring establish that interference invalidated observation 10. A coincident HVAC start alone would not prove causation. Preserve original data and follow the approved documented exclusion and replacement or repeat-run method. The nine-reading calculation below illustrates arithmetic after justified exclusion, not permission to trim an inconvenient result.

Step 5: Recalculate Statistics for the Valid Sample ( n = 9 n = 9 n = 9 )

Sum of the remaining 9 readings: ∑ x i = 90.00114 V \sum x_i = 90.00114\text{ V} ∑ x i ​ = 90.00114 V . x ˉ new = 90.00114 V 9 = 10.0001267 V ≈ 10.000127 V \bar{x}_{\text{new}} = \frac{90.00114\text{ V}}{9} = 10.0001267\text{ V} \approx 10.000127\text{ V} x ˉ new ​ = 9 90.00114 V ​ = 10.0001267 V ≈ 10.000127 V ∑ i = 1 9 ( x i − x ˉ new ) 2 = 4.400 × 10 − 9 V 2 \sum_{i=1}^9 (x_i - \bar{x}_{\text{new}})^2 = 4.400 \times 10^{-9}\text{ V}^2 i = 1 ∑ 9 ​ ( x i ​ − x ˉ new ​ ) 2 = 4.400 × 1 0 − 9 V 2 ν new = 9 − 1 = 8 \nu_{\text{new}} = 9 - 1 = 8 ν new ​ = 9 − 1 = 8 s new = 4.400 × 10 − 9 8 = 2.3452 × 10 − 5 V = 23.45 μ V s_{\text{new}} = \sqrt{\frac{4.400 \times 10^{-9}}{8}} = 2.3452 \times 10^{-5}\text{ V} = 23.45\ \mu\text{V} s new ​ = 8 4.400 × 1 0 − 9 ​ ​ = 2.3452 × 1 0 − 5 V = 23.45 μ V

Step 6: Compute Standard Uncertainty of the Mean u ( x ˉ ) u(\bar{x}) u ( x ˉ )

Because the calibration certificate will report the mean of the 9 readings as the calibrated voltage: u ( x ˉ ) = s new n = 23.45 μ V 9 = 23.45 μ V 3 = 7.82 μ V u(\bar{x}) = \frac{s_{\text{new}}}{\sqrt{n}} = \frac{23.45\ \mu\text{V}}{\sqrt{9}} = \frac{23.45\ \mu\text{V}}{3} = 7.82\ \mu\text{V} u ( x ˉ ) = n ​ s new ​ ​ = 9 ​ 23.45 μ V ​ = 3 23.45 μ V ​ = 7.82 μ V

Notice the metrological impact:

The sample standard deviation s = 23.45 μ V s = 23.45\ \mu\text{V} s = 23.45 μ V represents the process repeatability (dispersion of single voltmeter readings).

The standard uncertainty of the mean u ( x ˉ ) = 7.82 μ V u(\bar{x}) = 7.82\ \mu\text{V} u ( x ˉ ) = 7.82 μ V is the Type A standard uncertainty that enters the combined uncertainty budget for the reported mean value.

Official references (checked October 10, 2026): NIST Grubbs test definition and critical-value formula .