Study guide
7.2 Linear Regression and Residual Analysis
ASQ Certified Six Sigma Black Belt CSSBB
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Introduction & CSSBB Exam Strategy
1.1 CSSBB Exam Format, Scoring, and Testing Windows 1.2 Eligibility, Project Affidavits, Application, and Recertification 1.3 Open-Book Strategy, Reference Materials, and a 16-Week Study Plan
2 Chapter 2: Organization-Wide Planning and Deployment
2.1 Fundamentals of Six Sigma and Lean Methodologies 2.2 Selecting the Right Methodology: DMAIC, DFSS, and Lean 2.3 Business Systems, Processes, and Stakeholder Relationships 2.4 Strategic Planning and Deployment of Six Sigma Initiatives 2.5 Six Sigma Roles and Responsibilities 2.6 Organizational Barriers and Common Causes of Six Sigma Failure 2.7 Change Management: Stakeholder Analysis, Readiness, and Communication
3 Chapter 3: Organizational Process Management and Measures
3.1 Impact of Six Sigma Projects on Customers, Suppliers, and Stakeholders 3.2 Benchmarking Types and Performance Goal Setting 3.3 Performance Measures: Balanced Scorecard, KPIs, KBIs, OKRs, and Line of Sight 3.4 Leading and Lagging Indicators and Performance Dashboards 3.5 Financial Measures: Revenue Growth, Margin, NPV, ROI, and Cost-Benefit Analysis 3.6 Cost of Poor Quality and Hard versus Soft Savings
4 Chapter 4: Team Management
4.1 Team Types and Constraints 4.2 Team Roles, Responsibilities, and Member Selection 4.3 Team Success Factors and Ground Rules 4.4 Motivational Techniques and Overcoming Demotivators 4.5 Stages of Team Development and Conflict Resolution 4.6 Team Communication Plans 4.7 Team Leadership Models: Direct, Coach, Support, and Delegate 4.8 Group Behaviors, Conflict Resolution, and Meeting Management 4.9 Facilitation Techniques and Team Decision-Making Methods 4.10 Team Training: Needs Assessment, Delivery, and Evaluation
5 Chapter 5: Define Phase
5.1 Customer Identification and Segmentation 5.2 Customer Data Collection: Methods, Validity, and Reliability 5.3 Customer Requirements: CTX, CTQ Trees, and Operational Definitions 5.4 The Kano Model and Quality Function Deployment 5.5 SIPOC and Process Mapping 5.6 Business Case Justification 5.7 The Project Charter and the Problem Statement 5.8 Project Scope and Boundaries 5.9 SMART Goals and Project Performance Measurements 5.10 Project Charter Reviews and Toll-Gate Governance 5.11 Project Management Tools: Gantt Charts, Toll-Gate Reviews, WBS, and RACI 5.12 Analytical Tools Used Throughout the DMAIC Cycle
6 Chapter 6: Measure Phase
6.1 Process Flow Metrics and the Hidden Factory 6.2 Process Analysis Tools and Value Stream Mapping 6.3 Types of Data and Measurement Scales 6.4 Sampling Concepts, Methods, and Sample Size for Estimation 6.5 Data Collection Plans and Common Pitfalls 6.6 Gage R&R for Variable Measurement Systems 6.7 Attribute Agreement Analysis and Audit MSA 6.8 Measurement Systems Across the Organization, and Metrology 6.9 Population Parameters, Sample Statistics, and Valid Statistical Conclusions 6.10 Descriptive Statistics: Central Tendency, Dispersion, and Shape 6.11 Graphical Methods for Exploratory Data Analysis 6.12 Probability Concepts, Common Distributions, and the Central Limit Theorem 6.13 Additional Distributions: Hypergeometric, Exponential, Lognormal, Weibull, and Bivariate 6.14 Process Capability and Performance Indices 6.15 Capability Studies, Attributes Capability, Yield Metrics, and Long-Term Capability
7 Chapter 7: Analyze Phase
7.1 The Correlation Coefficient, Its Confidence Interval, and Causation 7.2 Linear Regression and Residual Analysis 7.3 Multivariate Tools: Factor Analysis, Discriminant Analysis, and MANOVA 7.4 Hypothesis Testing Terminology, Errors, and Single-Sample Tests 7.5 Statistical Versus Practical Significance 7.6 Sample Size for Hypothesis Tests 7.7 Point Estimates, Confidence, Prediction, and Tolerance Intervals 7.8 Tests for Means, Variances, and Proportions 7.9 Analysis of Variance 7.10 Chi-Square Goodness-of-Fit and Tests of Independence 7.11 Types of Risk and Failure Mode and Effects Analysis 7.12 Gap Analysis and Root Cause Analysis 7.13 Waste Analysis: The Seven Classic Wastes and Resource Under-Utilization
8 Chapter 8: Improve Phase
8.1 Design of Experiments: Objectives, Terminology, and Effects 8.2 DOE Design Principles: Randomization, Blocking, Replication, and Resolution 8.3 Planning and Evaluating Designed Experiments 8.4 One-Factor Experiments: Completely Randomized, Randomized Block, and Latin Square Designs 8.5 Full Factorial Experiments and Design Selection 8.6 Two-Level Fractional Factorial Experiments and Confounding 8.7 Response Surface Methodology and Optimization 8.8 Waste Elimination: Pull Systems, Kanban, 5S, and Poka-Yoke 8.9 Cycle-Time Reduction: Continuous Flow, SMED, and Heijunka 8.10 Kaizen, Kaizen Blitz, Theory of Constraints, and OEE 8.11 Implementation Planning, Piloting, and Solution Selection
9 Chapter 9: Control Phase
9.1 SPC Objectives, Variable Selection, and Rational Subgrouping 9.2 Control Charts for Variables Data 9.3 Control Charts for Attributes Data 9.4 Short-Run SPC and Moving Average Charts 9.5 Control Chart Analysis: Common Cause, Special Cause, and Run Rules 9.6 Total Productive Maintenance and Visual Controls 9.7 Measurement System Reanalysis 9.8 Control Plan Development and Process Owner Handoff 9.9 Lessons Learned, Documentation, and Training for Sustainment 9.10 Ongoing Evaluation of the Improved Process
10 Chapter 10: Design for Six Sigma (DFSS)
10.1 DFSS Frameworks: DMADV, DMADOV, and Design Transfer 10.2 Design for X (DFX) 10.3 Robust Design, Parameter Design, and Verification 10.4 Tolerance Design and Statistical Tolerancing
7.2 Linear Regression and Residual Analysis
Key Takeaways Explain This Topic AI
Pearson correlation coefficient r in [-1, +1] measures the strength and direction of linear association, whereas R^2 = r^2 represents the proportion of total variation in Y explained by X.
Simple linear regression models Y = beta0 + beta1 * X + epsilon, where least-squares slope is beta1-hat = Sxy / Sxx and intercept is beta0-hat = Y-bar - beta1-hat * X-bar.
Multicollinearity distorts regression coefficients and inflates standard errors; it is detected using Variance Inflation Factor VIF_j = 1 / (1 - R_j^2), where VIF > 5 to 10 indicates severe collinearity.
Adjusted R^2 penalizes model complexity by accounting for the number of predictors: R_adj^2 = 1 - [ (SS_res/(n-p)) / (SS_tot/(n-1)) ].
Regression residual validity relies on three core assumptions: normality of residuals (e_i ~ N(0, sigma^2)), homoscedasticity (constant variance), and independence (no autocorrelation).
Last updated: August 2026
Regression and correlation analysis are core statistical modeling tools utilized in the Analyze phase to quantify relationships between continuous process inputs ( X X X ) and continuous output metrics ( Y Y Y ). They enable Black Belts to establish empirical transfer functions Y = f ( X ) Y = f(X) Y = f ( X ) for process optimization and control.
Simple Linear Regression Modeling
Simple linear regression fits an empirical straight line to predict a continuous response variable Y Y Y from a single continuous predictor variable X X X : Y = β 0 + β 1 X + ϵ Y = \beta_0 + \beta_1 X + \epsilon Y = β 0 + β 1 X + ϵ
Where:
β 0 \beta_0 β 0 : Intercept parameter (value of Y Y Y when X = 0 X = 0 X = 0 ).
β 1 \beta_1 β 1 : Slope parameter (change in Y Y Y per unit increase in X X X ).
ϵ \epsilon ϵ : Random error term, assumed ϵ ∼ N ( 0 , σ 2 ) \epsilon \sim N(0, \sigma^2) ϵ ∼ N ( 0 , σ 2 ) .
Least Squares Estimation Formulas β ^ 1 = ∑ ( x i − x ˉ ) ( y i − y ˉ ) ∑ ( x i − x ˉ ) 2 = r ( s y s x ) \hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} = r \left( \frac{s_y}{s_x} \right) β ^ 1 = ∑ ( x i − x ˉ ) 2 ∑ ( x i − x ˉ ) ( y i − y ˉ ) = r ( s x s y ) β ^ 0 = y ˉ − β ^ 1 x ˉ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} β ^ 0 = y ˉ − β ^ 1 x ˉ
- Model Evaluation Metrics: R 2 R^2 R 2 and Adjusted R 2 R^2 R 2
- Coefficient of Determination ( R 2 R^2 R 2 )
Represents the proportion of total variation in the response variable Y Y Y explained by the regression model: R 2 = SS regression SS total = 1 − SS residual SS total R^2 = \frac{\text{SS}_{\text{regression}}}{\text{SS}_{\text{total}}} = 1 - \frac{\text{SS}_{\text{residual}}}{\text{SS}_{\text{total}}} R 2 = SS total SS regression = 1 − SS total SS residual
Adjusted R 2 R^2 R 2 ( R adj 2 R^2_{\text{adj}} R adj 2 )
Penalizes the model for adding unnecessary predictor variables that do not significantly improve fit: R adj 2 = 1 − [ ( 1 − R 2 ) ( n − 1 ) n − p − 1 ] R^2_{\text{adj}} = 1 - \left[ \frac{(1 - R^2)(n - 1)}{n - p - 1} \right] R adj 2 = 1 − [ n − p − 1 ( 1 − R 2 ) ( n − 1 ) ]
Where p p p is the number of predictor terms in the model.
Residual Analysis & Regression Assumptions Verification
Valid linear regression modeling requires checking four residual diagnostic assumptions ( LINE ):
Linearity : Residuals plot vs. fitted values shows a random scatter around zero without curvature.
Independence : Residuals show no autocorrelation across time or observation order (verified via Durbin-Watson statistic).
Normality : Residuals follow a Normal distribution (verified via Anderson-Darling test, p > 0.05 p > 0.05 p > 0.05 ).
Equal Variance (Homoscedasticity) : Residual spread remains constant across all predicted values of Y ^ \hat{Y} Y ^ .
Multiple Linear Regression & Multicollinearity
Multiple linear regression incorporates p ≥ 2 p \ge 2 p ≥ 2 continuous or dummy predictor variables: Y = β 0 + β 1 X 1 + β 2 X 2 + ⋯ + β p X p + ϵ Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_p X_p + \epsilon Y = β 0 + β 1 X 1 + β 2 X 2 + ⋯ + β p X p + ϵ
Multicollinearity & Variance Inflation Factor (VIF)
Multicollinearity occurs when predictor variables ( X i , X j X_i, X_j X i , X j ) are highly correlated with each other, inflating parameter standard errors. Multicollinearity is evaluated using the Variance Inflation Factor (VIF) : VIF i = 1 1 − R i 2 \text{VIF}_i = \frac{1}{1 - R_i^2} VIF i = 1 − R i 2 1
VIF < 5 < 5 < 5 : Acceptable low collinearity.
VIF ≥ 5 − 10 \ge 5 - 10 ≥ 5 − 10 : High collinearity; model parameters are unstable. Predictors must be centered or removed.
Hypothesis Testing for Regression Parameters
Once a linear regression model is fitted, statistical hypothesis tests evaluate whether regression parameters ( β 0 , β 1 \beta_0, \beta_1 β 0 , β 1 ) differ significantly from zero.
1. t t t -Test for Slope Significance ( β 1 \beta_1 β 1 )
Evaluates H 0 : β 1 = 0 H_0: \beta_1 = 0 H 0 : β 1 = 0 (no linear relationship) vs. H a : β 1 ≠ 0 H_a: \beta_1 \neq 0 H a : β 1 = 0 : t 0 = β ^ 1 SE ( β ^ 1 ) t_0 = \frac{\hat{\beta}_1}{\text{SE}(\hat{\beta}_1)} t 0 = SE ( β ^ 1 ) β ^ 1
Where standard error of slope is: SE ( β ^ 1 ) = s e ∑ ( x i − x ˉ ) 2 = MS residual SS x x \text{SE}(\hat{\beta}_1) = \frac{s_e}{\sqrt{\sum (x_i - \bar{x})^2}} = \frac{\sqrt{\text{MS}_{\text{residual}}}}{\sqrt{\text{SS}_{xx}}} SE ( β ^ 1 ) = ∑ ( x i − x ˉ ) 2 s e = SS xx MS residual
Decision Rule : If ∣ t 0 ∣ > t α / 2 , n − 2 |t_0| > t_{\alpha/2, n-2} ∣ t 0 ∣ > t α /2 , n − 2 ( p < α p < \alpha p < α ), reject H 0 H_0 H 0 . Predictor X X X significantly influences response Y Y Y .
2. Analysis of Variance (ANOVA) for Regression Model Significance
Evaluates overall model utility ( H 0 : β 1 = β 2 = ⋯ = β p = 0 H_0: \beta_1 = \beta_2 = \dots = \beta_p = 0 H 0 : β 1 = β 2 = ⋯ = β p = 0 ):
Source of Variation Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F F F -Statistic ( F 0 F_0 F 0 )
Regression (Model) SS reg = ∑ ( y ^ i − y ˉ ) 2 \text{SS}_{\text{reg}} = \sum (\hat{y}_i - \bar{y})^2 SS reg = ∑ ( y ^ i − y ˉ ) 2 p p p MS reg = SS reg p \text{MS}_{\text{reg}} = \frac{\text{SS}_{\text{reg}}}{p} MS reg = p SS reg F 0 = MS reg MS res F_0 = \frac{\text{MS}_{\text{reg}}}{\text{MS}_{\text{res}}} F 0 = MS res MS reg
Residual (Error) SS res = ∑ ( y i − y ^ i ) 2 \text{SS}_{\text{res}} = \sum (y_i - \hat{y}_i)^2 SS res = ∑ ( y i − y ^ i ) 2 n − p − 1 n - p - 1 n − p − 1 MS res = SS res n − p − 1 \text{MS}_{\text{res}} = \frac{\text{SS}_{\text{res}}}{n - p - 1} MS res = n − p − 1 SS res
Total SS total = ∑ ( y i − y ˉ ) 2 \text{SS}_{\text{total}} = \sum (y_i - \bar{y})^2 SS total = ∑ ( y i − y ˉ ) 2 n − 1 n - 1 n − 1
Worked Simple Linear Regression Example
An chemical engineer models reactor yield Y Y Y (%) against reaction temperature X X X ( ∘ C ^\circ\text{C} ∘ C ). Data from n = 10 n = 10 n = 10 pilot runs yields:
x ˉ = 150.0 ∘ C \bar{x} = 150.0^\circ\text{C} x ˉ = 150. 0 ∘ C , y ˉ = 82.0 % \bar{y} = 82.0\% y ˉ = 82.0%
SS x x = ∑ ( x i − x ˉ ) 2 = 2 , 000.0 \text{SS}_{xx} = \sum (x_i - \bar{x})^2 = 2,000.0 SS xx = ∑ ( x i − x ˉ ) 2 = 2 , 000.0
SS x y = ∑ ( x i − x ˉ ) ( y i − y ˉ ) = 3 , 600.0 \text{SS}_{xy} = \sum (x_i - \bar{x})(y_i - \bar{y}) = 3,600.0 SS x y = ∑ ( x i − x ˉ ) ( y i − y ˉ ) = 3 , 600.0
SS y y = ∑ ( y i − y ˉ ) 2 = 6 , 800.0 \text{SS}_{yy} = \sum (y_i - \bar{y})^2 = 6,800.0 SS yy = ∑ ( y i − y ˉ ) 2 = 6 , 800.0
1. Calculate Regression Slope & Intercept β ^ 1 = SS x y SS x x = 3 , 600.0 2 , 000.0 = 1.80 % / ∘ C \hat{\beta}_1 = \frac{\text{SS}_{xy}}{\text{SS}_{xx}} = \frac{3,600.0}{2,000.0} = 1.80\%/^\circ\text{C} β ^ 1 = SS xx SS x y = 2 , 000.0 3 , 600.0 = 1.80% / ∘ C β ^ 0 = y ˉ − β ^ 1 x ˉ = 82.0 − ( 1.80 × 150.0 ) = 82.0 − 270.0 = − 188.0 % \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} = 82.0 - (1.80 \times 150.0) = 82.0 - 270.0 = -188.0\% β ^ 0 = y ˉ − β ^ 1 x ˉ = 82.0 − ( 1.80 × 150.0 ) = 82.0 − 270.0 = − 188.0%
Fitted Regression Model : Y ^ = − 188.0 + 1.80 X \hat{Y} = -188.0 + 1.80 X Y ^ = − 188.0 + 1.80 X
2. Calculate Coefficient of Determination ( R 2 R^2 R 2 ) SS reg = β ^ 1 SS x y = 1.80 × 3 , 600.0 = 6 , 480.0 \text{SS}_{\text{reg}} = \hat{\beta}_1 \text{SS}_{xy} = 1.80 \times 3,600.0 = 6,480.0 SS reg = β ^ 1 SS x y = 1.80 × 3 , 600.0 = 6 , 480.0 R 2 = SS reg SS yy = 6 , 480.0 6 , 800.0 = 0.9529 ⟹ 95.29 % R^2 = \frac{\text{SS}_{\text{reg}}}{\text{SS}_{\text{yy}}} = \frac{6,480.0}{6,800.0} = 0.9529 \implies 95.29\% R 2 = SS yy SS reg = 6 , 800.0 6 , 480.0 = 0.9529 ⟹ 95.29%
Interpretation : 95.29 % 95.29\% 95.29% of the variation in reactor yield is explained by reaction temperature.
Prediction Intervals vs. Confidence Intervals
When predicting Y Y Y at a specific predictor setting X 0 X_0 X 0 :
Confidence Interval for Mean Response : Quantifies uncertainty in estimating the expected mean response E ( Y ∣ X 0 ) E(Y|X_0) E ( Y ∣ X 0 ) : y ^ 0 ± t α / 2 , n − 2 s e 1 n + ( x 0 − x ˉ ) 2 SS x x \hat{y}_0 \pm t_{\alpha/2, n-2} \, s_e \sqrt{\frac{1}{n} + \frac{(x_0 - \bar{x})^2}{\text{SS}_{xx}}} y ^ 0 ± t α /2 , n − 2 s e n 1 + SS xx ( x 0 − x ˉ ) 2
Prediction Interval for Single Future Observation : Quantifies uncertainty in predicting a single future individual value Y 0 Y_0 Y 0 . It is significantly wider because it incorporates individual observation error: y ^ 0 ± t α / 2 , n − 2 s e 1 + 1 n + ( x 0 − x ˉ ) 2 SS x x \hat{y}_0 \pm t_{\alpha/2, n-2} \, s_e \sqrt{1 + \frac{1}{n} + \frac{(x_0 - \bar{x})^2}{\text{SS}_{xx}}} y ^ 0 ± t α /2 , n − 2 s e 1 + n 1 + SS xx ( x 0 − x ˉ ) 2