7.9 Analysis of Variance

ASQ Certified Six Sigma Black Belt CSSBB

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & CSSBB Exam Strategy

1.1 CSSBB Exam Format, Scoring, and Testing Windows 1.2 Eligibility, Project Affidavits, Application, and Recertification 1.3 Open-Book Strategy, Reference Materials, and a 16-Week Study Plan

2 Chapter 2: Organization-Wide Planning and Deployment

2.1 Fundamentals of Six Sigma and Lean Methodologies 2.2 Selecting the Right Methodology: DMAIC, DFSS, and Lean 2.3 Business Systems, Processes, and Stakeholder Relationships 2.4 Strategic Planning and Deployment of Six Sigma Initiatives 2.5 Six Sigma Roles and Responsibilities 2.6 Organizational Barriers and Common Causes of Six Sigma Failure 2.7 Change Management: Stakeholder Analysis, Readiness, and Communication

3 Chapter 3: Organizational Process Management and Measures

3.1 Impact of Six Sigma Projects on Customers, Suppliers, and Stakeholders 3.2 Benchmarking Types and Performance Goal Setting 3.3 Performance Measures: Balanced Scorecard, KPIs, KBIs, OKRs, and Line of Sight 3.4 Leading and Lagging Indicators and Performance Dashboards 3.5 Financial Measures: Revenue Growth, Margin, NPV, ROI, and Cost-Benefit Analysis 3.6 Cost of Poor Quality and Hard versus Soft Savings

4 Chapter 4: Team Management

4.1 Team Types and Constraints 4.2 Team Roles, Responsibilities, and Member Selection 4.3 Team Success Factors and Ground Rules 4.4 Motivational Techniques and Overcoming Demotivators 4.5 Stages of Team Development and Conflict Resolution 4.6 Team Communication Plans 4.7 Team Leadership Models: Direct, Coach, Support, and Delegate 4.8 Group Behaviors, Conflict Resolution, and Meeting Management 4.9 Facilitation Techniques and Team Decision-Making Methods 4.10 Team Training: Needs Assessment, Delivery, and Evaluation

5 Chapter 5: Define Phase

5.1 Customer Identification and Segmentation 5.2 Customer Data Collection: Methods, Validity, and Reliability 5.3 Customer Requirements: CTX, CTQ Trees, and Operational Definitions 5.4 The Kano Model and Quality Function Deployment 5.5 SIPOC and Process Mapping 5.6 Business Case Justification 5.7 The Project Charter and the Problem Statement 5.8 Project Scope and Boundaries 5.9 SMART Goals and Project Performance Measurements 5.10 Project Charter Reviews and Toll-Gate Governance 5.11 Project Management Tools: Gantt Charts, Toll-Gate Reviews, WBS, and RACI 5.12 Analytical Tools Used Throughout the DMAIC Cycle

6 Chapter 6: Measure Phase

6.1 Process Flow Metrics and the Hidden Factory 6.2 Process Analysis Tools and Value Stream Mapping 6.3 Types of Data and Measurement Scales 6.4 Sampling Concepts, Methods, and Sample Size for Estimation 6.5 Data Collection Plans and Common Pitfalls 6.6 Gage R&R for Variable Measurement Systems 6.7 Attribute Agreement Analysis and Audit MSA 6.8 Measurement Systems Across the Organization, and Metrology 6.9 Population Parameters, Sample Statistics, and Valid Statistical Conclusions 6.10 Descriptive Statistics: Central Tendency, Dispersion, and Shape 6.11 Graphical Methods for Exploratory Data Analysis 6.12 Probability Concepts, Common Distributions, and the Central Limit Theorem 6.13 Additional Distributions: Hypergeometric, Exponential, Lognormal, Weibull, and Bivariate 6.14 Process Capability and Performance Indices 6.15 Capability Studies, Attributes Capability, Yield Metrics, and Long-Term Capability

7 Chapter 7: Analyze Phase

7.1 The Correlation Coefficient, Its Confidence Interval, and Causation 7.2 Linear Regression and Residual Analysis 7.3 Multivariate Tools: Factor Analysis, Discriminant Analysis, and MANOVA 7.4 Hypothesis Testing Terminology, Errors, and Single-Sample Tests 7.5 Statistical Versus Practical Significance 7.6 Sample Size for Hypothesis Tests 7.7 Point Estimates, Confidence, Prediction, and Tolerance Intervals 7.8 Tests for Means, Variances, and Proportions 7.9 Analysis of Variance 7.10 Chi-Square Goodness-of-Fit and Tests of Independence 7.11 Types of Risk and Failure Mode and Effects Analysis 7.12 Gap Analysis and Root Cause Analysis 7.13 Waste Analysis: The Seven Classic Wastes and Resource Under-Utilization

8 Chapter 8: Improve Phase

8.1 Design of Experiments: Objectives, Terminology, and Effects 8.2 DOE Design Principles: Randomization, Blocking, Replication, and Resolution 8.3 Planning and Evaluating Designed Experiments 8.4 One-Factor Experiments: Completely Randomized, Randomized Block, and Latin Square Designs 8.5 Full Factorial Experiments and Design Selection 8.6 Two-Level Fractional Factorial Experiments and Confounding 8.7 Response Surface Methodology and Optimization 8.8 Waste Elimination: Pull Systems, Kanban, 5S, and Poka-Yoke 8.9 Cycle-Time Reduction: Continuous Flow, SMED, and Heijunka 8.10 Kaizen, Kaizen Blitz, Theory of Constraints, and OEE 8.11 Implementation Planning, Piloting, and Solution Selection

9 Chapter 9: Control Phase

9.1 SPC Objectives, Variable Selection, and Rational Subgrouping 9.2 Control Charts for Variables Data 9.3 Control Charts for Attributes Data 9.4 Short-Run SPC and Moving Average Charts 9.5 Control Chart Analysis: Common Cause, Special Cause, and Run Rules 9.6 Total Productive Maintenance and Visual Controls 9.7 Measurement System Reanalysis 9.8 Control Plan Development and Process Owner Handoff 9.9 Lessons Learned, Documentation, and Training for Sustainment 9.10 Ongoing Evaluation of the Improved Process

10 Chapter 10: Design for Six Sigma (DFSS)

10.1 DFSS Frameworks: DMADV, DMADOV, and Design Transfer 10.2 Design for X (DFX) 10.3 Robust Design, Parameter Design, and Verification 10.4 Tolerance Design and Statistical Tolerancing

7.9 Analysis of Variance

Key Takeaways Explain This Topic AI

ANOVA partitions total sum of squares into between-groups (treatment) and within-groups (error) components.

The F statistic is the ratio of mean square treatment to mean square error, and a large F indicates that group means differ.

ANOVA controls the family-wise error rate that repeated pairwise t-tests would inflate.

A significant F test says that at least one mean differs but not which; post-hoc tests such as Tukey's HSD identify the pairs.

ANOVA assumes independence, normality of residuals, and homogeneity of variances, and is most sensitive to violations of independence.

Last updated: August 2026

Why not repeated t-tests

With k k k groups there are k ( k − 1 ) / 2 k(k-1)/2 k ( k − 1 ) /2 pairwise comparisons. For five groups that is ten tests, and at α = 0.05 \alpha = 0.05 α = 0.05 per test the family-wise error rate is 1 − 0.95 10 = 0.40 1 - 0.95^{10} = 0.40 1 − 0.9 5 10 = 0.40 . ANOVA answers the

question with a single test at the stated α \alpha α .

The partition S S t o t a l = S S t r e a t m e n t + S S e r r o r SS_{total} = SS_{treatment} + SS_{error} S S t o t a l ​ = S S t re a t m e n t ​ + S S error ​ S S t o t a l = ∑ i ∑ j ( y i j − y ˉ ) 2 , S S t r e a t = ∑ i n i ( y ˉ i − y ˉ ) 2 , S S e r r o r = ∑ i ∑ j ( y i j − y ˉ i ) 2 SS_{total} = \sum_{i}\sum_{j}(y_{ij} - \bar{y})^2, \qquad SS_{treat} = \sum_{i} n_i(\bar{y}_i - \bar{y})^2, \qquad SS_{error} = \sum_{i}\sum_{j}(y_{ij} - \bar{y}_i)^2 S S t o t a l ​ = i ∑ ​ j ∑ ​ ( y ij ​ − y ˉ ​ ) 2 , S S t re a t ​ = i ∑ ​ n i ​ ( y ˉ ​ i ​ − y ˉ ​ ) 2 , S S error ​ = i ∑ ​ j ∑ ​ ( y ij ​ − y ˉ ​ i ​ ) 2

• The logic: if the group means were all equal, the only reason group averages would differ is random
• error, so between-group variation would be about the same size as within-group variation. If treatment

means genuinely differ, between-group variation is inflated.

The ANOVA table

Source Sum of squares df Mean square F

Treatment (between) S S t r e a t SS_{treat} S S t re a t ​ k − 1 k - 1 k − 1 M S t r e a t = S S t r e a t / ( k − 1 ) MS_{treat} = SS_{treat}/(k-1) M S t re a t ​ = S S t re a t ​ / ( k − 1 ) M S t r e a t / M S e r r o r MS_{treat}/MS_{error} M S t re a t ​ / M S error ​

Error (within) S S e r r o r SS_{error} S S error ​ N − k N - k N − k M S e r r o r = S S e r r o r / ( N − k ) MS_{error} = SS_{error}/(N-k) M S error ​ = S S error ​ / ( N − k )

Total S S t o t a l SS_{total} S S t o t a l ​ N − 1 N - 1 N − 1

where k k k is the number of groups and N N N the total number of observations. Reject H 0 : μ 1 = μ 2 = ⋯ = μ k H_0: \mu_1 = \mu_2 = \cdots = \mu_k H 0 ​ : μ 1 ​ = μ 2 ​ = ⋯ = μ k ​ when F > F α , k − 1 , N − k F > F_{\alpha,\,k-1,\,N-k} F > F α , k − 1 , N − k ​ .

Note that M S e r r o r MS_{error} M S error ​ is the pooled estimate of the common within-group variance σ 2 \sigma^2 σ 2 , and its

square root is the standard deviation used for post-hoc comparisons.

Worked example

Four suppliers, five specimens each, tensile strength in MPa.

• Supplier Observations y ˉ i \bar{y}_i y ˉ ​ i ​
• A 412, 418, 415, 421, 414 416.0
• B 428, 431, 425, 433, 428 429.0
• C 409, 414, 411, 407, 414 411.0
• D 424, 419, 427, 422, 423 423.0

Grand mean y ˉ = 419.75 \bar{y} = 419.75 y ˉ ​ = 419.75 , N = 20 N = 20 N = 20 , k = 4 k = 4 k = 4 . S S t r e a t = 5 [ ( 416.0 − 419.75 ) 2 + ( 429.0 − 419.75 ) 2 + ( 411.0 − 419.75 ) 2 + ( 423.0 − 419.75 ) 2 ] SS_{treat} = 5\left[(416.0 - 419.75)^2 + (429.0 - 419.75)^2 + (411.0 - 419.75)^2 + (423.0 - 419.75)^2\right] S S t re a t ​ = 5 [ ( 416.0 − 419.75 ) 2 + ( 429.0 − 419.75 ) 2 + ( 411.0 − 419.75 ) 2 + ( 423.0 − 419.75 ) 2 ] = 5 [ 14.06 + 85.56 + 76.56 + 10.56 ] = 5 × 186.75 = 933.75 = 5\left[14.06 + 85.56 + 76.56 + 10.56\right] = 5 \times 186.75 = 933.75 = 5 [ 14.06 + 85.56 + 76.56 + 10.56 ] = 5 × 186.75 = 933.75

Within-group sums of squares are 50.0, 38.0, 38.0, and 34.0, giving S S e r r o r = 160.0 SS_{error} = 160.0 S S error ​ = 160.0 .

• Source SS df MS F
• Supplier 933.75 3 311.25 31.13
• Error 160.00 16 10.00
• Total 1093.75 19

F 0.05 , 3 , 16 = 3.24 F_{0.05,3,16} = 3.24 F 0.05 , 3 , 16 ​ = 3.24 , and 31.13 ≫ 3.24 31.13 \gg 3.24 31.13 ≫ 3.24 , so at least one supplier mean differs. The effect size is η 2 = S S t r e a t S S t o t a l = 933.75 1093.75 = 0.854 \eta^2 = \frac{SS_{treat}}{SS_{total}} = \frac{933.75}{1093.75} = 0.854 η 2 = S S t o t a l ​ S S t re a t ​ ​ = 1093.75 933.75 ​ = 0.854

so supplier accounts for 85.4% of the variation in tensile strength.

• After a significant F
• A significant F F F says at least one mean differs; it does not say which. Post-hoc procedures control

the family-wise error rate while making the pairwise comparisons:

• Procedure Use
• Tukey's HSD All pairwise comparisons; the standard default
• Dunnett's test Comparing several treatments against one control only
• Fisher's LSD Least conservative; only defensible after a significant F, and even then criticized
• Bonferroni Simple and conservative; adjust α \alpha α to α / m \alpha/m α / m for m m m comparisons
• Scheffe Most conservative; permits arbitrary contrasts decided after seeing the data
• Running post-hoc tests without a significant omnibus F reintroduces the error-rate inflation that

ANOVA exists to prevent.

Assumptions

• Assumption How to check Effect of violation
• Independent observations Study design; residuals versus run order Serious; invalidates the test

Residuals normally distributed Normal probability plot of residuals; Anderson-Darling Mild; ANOVA is robust with balanced designs and moderate n

Equal variances across groups Levene's or Bartlett's test; residuals versus fits Moderate; serious with unequal group sizes

• ANOVA is robust to moderate non-normality when the design is balanced, but it is not robust to
• dependence . Time-ordered data with autocorrelation, or observations clustered within batches, violate

the assumption in a way no transformation fixes; the design must change.

Bartlett's test for equal variances is itself very sensitive to non-normality, so Levene's test is

preferred in practice. Where variances differ materially, use Welch's ANOVA or transform the response.

When normality fails badly, the Kruskal-Wallis test is the non-parametric alternative.

Residual analysis is the practical way to check all three at once: plot residuals against fitted values

(equal variance), on a normal probability plot (normality), and against run order (independence).

Beyond one-way

Design Question

One-way Does one factor with k k k levels affect the response?

Two-way Do two factors affect the response, and do they interact?

Randomized block Does the factor matter, after removing a known nuisance source such as batch or day?

ANOVA within DOE Which main effects and interactions in a factorial design are significant?

• The two-way case introduces the interaction term, which tests whether the effect of one factor
• depends on the level of the other. Interactions are frequently the most important finding in a Six Sigma
• analysis, because they are invisible to one-factor-at-a-time experimentation. This is the direct link

between ANOVA and the Improve phase: the analysis of a factorial experiment is an ANOVA.