Study guide

10.2 Simple Linear Regression & Coefficient of Determination (R^2)

CSSC Certified Six Sigma Green Belt Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Six Sigma Overview & Exam Fundamentals

1.1 CSSC Certification Overview, Exam Structure & Body of Knowledge 1.2 History & Evolution of Six Sigma 1.3 Six Sigma Organizational Roles & Belts Hierarchy 1.4 The DMAIC Methodology Architecture vs. DMADV / DFSS 1.5 Other Process Improvement & Quality Methods, and When to Choose Six Sigma

2 Chapter 2: Quality Fundamentals & Defect Metrics

2.1 Critical to Quality (CTQ) Characteristics & VOC Translation 2.2 Defect Metrics: DPU, DPO, DPMO, & the 1.5-Sigma Shift 2.3 First Time Yield (FTY) & Rolled Throughput Yield (RTY) 2.4 Cost of Quality (COQ) & Cost of Poor Quality (COPQ)

3 Chapter 3: Lean Principles & Waste Elimination

3.1 Foundations of Lean & The Toyota Production System (TPS) 3.2 The Eight Wastes: CSSC's Seven Muda, TIMWOODS, and Muri/Mura 3.3 The 5S Visual Workplace Methodology 3.4 Value Stream Mapping (VSM), Cycle Time & Takt Time

4 Chapter 4: Project Selection & Team Management

4.1 Six Sigma Project Selection & Business Case Development 4.2 Project Charter Architecture 4.3 Stakeholder Management & Communication Planning 4.4 Team Dynamics & Tuckman's Team Development Stages 4.5 The 15-Point Project Viability Model & Process-Level Selection

5 Chapter 5: Define Phase & Process Mapping

5.1 High-Level Process Mapping with SIPOC 5.2 Flowcharts, Swimlane Diagrams & Process Value Analysis 5.3 Quality Function Deployment (QFD) & The House of Quality 5.4 Critical to Quality (CTQ) Flowdown & Operational Definitions 5.5 The CSSC Define Toolset: Ground Rules, In/Out of the Box, Is–Is Not & Tollgate Checklists

6 Chapter 6: Measure Phase & Measurement Systems

6.1 Data Types: Continuous (Variable) vs. Discrete (Attribute) Data 6.2 Data Collection Planning & Sampling Strategies 6.3 Measurement System Analysis (MSA) Foundations & Calibration 6.4 Variable & Attribute Gage R&R Studies

7 Chapter 7: Analyze Phase & Root Cause Analysis

7.1 Qualitative Root Cause Tools: 5 Whys & Cause-and-Effect (Fishbone) Diagrams 7.2 Failure Mode and Effects Analysis (FMEA) & RPN Calculation 7.3 Pareto Analysis & The 80/20 Rule 7.4 Graphical Data Exploration: Histograms, Box Plots, & Multi-Vari Charts 7.5 The Root Cause Verification Matrix

8 Chapter 8: Descriptive Statistics & Distributions

8.1 Measures of Central Tendency & Dispersion 8.2 The Normal Distribution & The Empirical Rule (68-95-99.7) 8.3 Standard Normal Distribution, Z-Scores & Probability Calculations 8.4 Common Non-Normal Distributions: Binomial, Poisson & Chi-Square

9 Chapter 9: Inferential Statistics & Hypothesis Testing

9.1 Fundamentals of Hypothesis Testing: Null vs. Alternative, Alpha & Beta Risks 9.2 One-Sample and Two-Sample t-Tests for Means 9.3 Paired t-Tests & Non-Parametric Alternatives 9.4 Analysis of Variance (One-Way & Two-Way ANOVA) & Chi-Square Contingency Tests 9.5 Sample Size Determination for Hypothesis Tests

10 Chapter 10: Correlation, Regression, & Experimentation

10.1 Linear Correlation & Pearson's Correlation Coefficient (r) 10.2 Simple Linear Regression & Coefficient of Determination (R^2) 10.3 Introduction to Design of Experiments (DOE) & Factorial Designs 10.4 Analyzing Experimental Results: Main Effects & Interaction Plots

11 Chapter 11: Improve Phase & Solution Implementation

11.1 Brainstorming, Creative Thinking & Solution Selection Matrices 11.2 Piloting Solutions & Conducting Risk Assessments 11.3 Implementation Planning & Change Management (ADKAR) 11.4 Mistake-Proofing (Poka-Yoke) & Standard Work

12 Chapter 12: Control Phase, SPC, & Process Capability

12.1 Statistical Process Control (SPC) & Shewhart Control Rules 12.2 Variable Control Charts: I-MR, X-bar & R, X-bar & S 12.3 Attribute Control Charts: p, np, c, and u Charts 12.4 Process Capability & Performance Indices: Cp, Cpk, Pp, Ppk 12.5 Communicating Statistical Results to Business Audiences

10.2 Simple Linear Regression & Coefficient of Determination (R^2)

Key Takeaways Explain This Topic AI

Simple Linear Regression establishes an operational transfer function, ŷ = b₀ + b₁x, that predicts continuous response Y from continuous input X by minimizing squared vertical residuals via Ordinary Least Squares (OLS).

The slope coefficient (b₁ = SS_xy / SS_xx = r(s_y / s_x)) quantifies the expected change in response variable Y for every 1-unit increase in predictor X, while the intercept (b₀ = ȳ - b₁x̄) estimates the baseline response when X = 0.

Model evaluation requires deconstructing the software ANOVA table (SS_total = SS_reg + SS_error), testing model significance via the F-statistic (where F = t² in simple linear regression), and assessing R² alongside adjusted R².

Model validity relies strictly on verifying the four LINE assumptions: Linearity, Independence, Normality, and Equal Variance (Homoscedasticity) through comprehensive Four-in-One residual diagnostic analysis.

Extrapolating regression predictions beyond the observed experimental range of predictor X introduces severe operational risk because underlying physical relationships frequently become non-linear or undergo catastrophic regime shifts.

Last updated: September 2026

10.2 Simple Linear Regression & Coefficient of Determination (R^2)

Quick Summary: While correlation analysis quantifies the strength of association between two continuous variables, Simple Linear Regression establishes an operational transfer function—a mathematical equation expressing Y = f ( X ) Y = f(X) Y = f ( X ) . Using the Ordinary Least Squares (OLS) method, regression derives a best-fit straight line that minimizes the sum of squared vertical deviations between actual observed data and model predictions. Six Sigma Green Belts must be proficient in reading software regression outputs, interpreting slope ( b 1 b_1 b 1 ​ ) and intercept ( b 0 b_0 b 0 ​ ) parameters, deconstructing the ANOVA table , and validating the four foundational assumptions— LINE (Linearity, Independence, Normality, Equal Variance) —via residual analysis before deploying any model for process control or optimization.

The Transition from Correlation to Regression

In DMAIC projects, correlation answers: "Are X X X and Y Y Y linearly related?" Regression takes the next operational step and answers:

  • "By how many units does Y Y Y change when we adjust X X X by 1 unit?"
  • "What is the expected value of Y Y Y if we operate X X X at a specific target setting?"
  • "Can we mathematically control the process output by dialing in the input?" Correlation vs. Regression
  • CORRELATION (Bidirectional) REGRESSION (Directional Transfer Function)
  • ┌─────────────────────────┐ ┌─────────────────────────┐
  • │ Variable X ◀──▶ Variable Y│ │ Input Predictor (X) │
  • └─────────────────────────┘ └────────────┬────────────┘
  • Non-directional (r_xy = r_yx) │ Transfer Function
  • Scaled from -1.0 to +1.0 ▼ Y = b0 + b1(X)
  • Measures strength/association ┌─────────────────────────┐
  • No cause-and-effect equation │ Output Response (Y) │
  • └─────────────────────────┘
  • Directional (X predicts Y)
  • Retains physical measurement units
  • Quantifies magnitude of change (slope)
  • The Mathematical Model of Simple Linear Regression
  • The Theoretical Population Model

The theoretical population model assumes that for any specific value of the independent predictor variable X X X , the corresponding values of the dependent response variable Y Y Y are normally distributed around a linear mean line: Y = β 0 + β 1 X + ϵ Y = \beta_0 + \beta_1 X + \epsilon Y = β 0 ​ + β 1 ​ X + ϵ

Where:

  • Y Y Y is the dependent response variable (the process output / CTQ),
  • X X X is the independent predictor variable (the process input factor),
  • β 0 \beta_0 β 0 ​ (beta-zero) is the true population intercept (the value of Y Y Y when X = 0 X = 0 X = 0 ),

β 1 \beta_1 β 1 ​ (beta-one) is the true population slope (the rate of change in Y Y Y per unit change in X X X ),

ϵ \epsilon ϵ (epsilon) is the random unobserved error term, assumed to be independent and identically distributed as N ( 0 , σ 2 ) N(0, \sigma^2) N ( 0 , σ 2 ) .

The Sample Estimated Regression Line

Because true population parameters ( β 0 , β 1 ) (\beta_0, \beta_1) ( β 0 ​ , β 1 ​ ) are unobservable, Green Belts calculate sample estimates ( b 0 , b 1 ) (b_0, b_1) ( b 0 ​ , b 1 ​ ) from experimental sample data, yielding the sample prediction equation : y ^ = b 0 + b 1 x \hat{y} = b_0 + b_1 x y ^ ​ = b 0 ​ + b 1 ​ x

Where:

  • y ^ \hat{y} y ^ ​ (y-hat) is the fitted (predicted) value of Y Y Y for a given value of x x x ,
  • b 0 b_0 b 0 ​ is the sample estimate of the intercept,

b 1 b_1 b 1 ​ is the sample estimate of the slope.

The Ordinary Least Squares (OLS) Method

How do we determine which straight line among infinitely many possibilities represents the "best fit"? The standard methodology is Ordinary Least Squares (OLS) .

Defining the Residual (Error)

For each observed data point ( x i , y i ) (x_i, y_i) ( x i ​ , y i ​ ) , the vertical distance between the actual observed value ( y i y_i y i ​ ) and the predicted value on the regression line ( y ^ i \hat{y}_i y ^ ​ i ​ ) is called the residual ( e i e_i e i ​ ): Residual e i = y i − y ^ i = y i − ( b 0 + b 1 x i ) \text{Residual } e_i = y_i - \hat{y}_i = y_i - (b_0 + b_1 x_i) Residual e i ​ = y i ​ − y ^ ​ i ​ = y i ​ − ( b 0 ​ + b 1 ​ x i ​ ) The Ordinary Least Squares Concept

  • Y ▲
  • │ (x_i, y_i) Actual Point
  • │ █
  • │ │ ▲
  • │ │ │ Residual e_i = y_i - ŷ_i
  • │ │ ▼ (Vertical Distance)
  • │ ▲ ○ ŷ_i (Point on Line)
  • │ ╱ │ e_j
  • │ ╱ ▼ █
  • │ ╱
  • │ ╱ Regression Line: ŷ = b0 + b1(x)
  • │ Intercept b0 ╱ (Minimizes Sum of Squared e_i²)
  • │─────────────○─────
  • │
  • └─────────────────────────────────────────────────────────────► X
  • 0
  • The Minimization Criterion

If we simply summed the raw residuals ( ∑ e i \sum e_i ∑ e i ​ ), positive deviations above the line would cancel out negative deviations below the line, resulting in zero for countless arbitrary lines. To eliminate signs and penalize larger errors, OLS squares every residual and minimizes the Sum of Squared Errors ( S S E SSE SSE ) : Minimize S S E = ∑ i = 1 n e i 2 = ∑ i = 1 n ( y i − y ^ i ) 2 = ∑ i = 1 n [ y i − ( b 0 + b 1 x i ) ] 2 \text{Minimize } SSE = \sum_{i=1}^n e_i^2 = \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \sum_{i=1}^n [y_i - (b_0 + b_1 x_i)]^2 Minimize SSE = i = 1 ∑ n ​ e i 2 ​ = i = 1 ∑ n ​ ( y i ​ − y ^ ​ i ​ ) 2 = i = 1 ∑ n ​ [ y i ​ − ( b 0 ​ + b 1 ​ x i ​ ) ] 2

The OLS Formulas for Slope and Intercept

Using calculus (taking partial derivatives with respect to b 0 b_0 b 0 ​ and b 1 b_1 b 1 ​ and setting them to zero), the unique least-squares estimators are derived: b 1 = S S x y S S x x = ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) ∑ i = 1 n ( x i − x ˉ ) 2 = r ( s y s x ) b_1 = \frac{SS_{xy}}{SS_{xx}} = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^n (x_i - \bar{x})^2} = r \left(\frac{s_y}{s_x}\right) b 1 ​ = S S xx ​ S S x y ​ ​ = ∑ i = 1 n ​ ( x i ​ − x ˉ ) 2 ∑ i = 1 n ​ ( x i ​ − x ˉ ) ( y i ​ − y ˉ ​ ) ​ = r ( s x ​ s y ​ ​ ) b 0 = y ˉ − b 1 x ˉ b_0 = \bar{y} - b_1 \bar{x} b 0 ​ = y ˉ ​ − b 1 ​ x ˉ

Key Mathematical Properties of the OLS Line

The Centroid Property : The regression line always passes through the point of sample means ( x ˉ , y ˉ ) (\bar{x}, \bar{y}) ( x ˉ , y ˉ ​ ) .

Zero Sum of Residuals : The sum (and arithmetic mean) of the OLS residuals is exactly zero: ∑ e i = 0 \sum e_i = 0 ∑ e i ​ = 0 .

Relationship to Correlation : The slope b 1 b_1 b 1 ​ and Pearson's r r r always share the same algebraic sign ( b 1 > 0 ⟺ r > 0 b_1 > 0 \iff r > 0 b 1 ​ > 0 ⟺ r > 0 ). However, while r r r is bounded by ± 1.0 \pm 1.0 ± 1.0 , the slope b 1 b_1 b 1 ​ has no bounds and reflects the actual units of Y / X Y / X Y / X .

Reading Software Regression Output & The ANOVA Table

In modern Six Sigma practice, regression is computed using statistical software (Minitab, JMP, or R). Green Belts must be fluent in deconstructing software output tables.

Sample Software Output: Curing Oven Analysis

Consider an industrial study evaluating heating time ( X X X , minutes) versus resin hardness ( Y Y Y , Shore D units) across n = 20 n = 20 n = 20 production batches: Regression Analysis: Hardness (Y) versus Heating Time (X)

The regression equation is:

Hardness = 22.41 + 3.125 Heating Time

  • Predictor Coef SE Coef T P
  • Constant 22.410 2.150 10.42 <0.001
  • Time (X) 3.125 0.285 10.96 <0.001

S = 2.4512 R-Sq = 87.0% R-Sq(adj) = 86.2%

  • Analysis of Variance (ANOVA)
  • Source DF SS MS F P
  • Regression 1 721.25 721.25 120.12 <0.001
  • Error (Residual) 18 108.08 6.00
  • Total 19 829.33
  • Deconstructing the Parameter Estimates Table

Coefficients ( Coef ) :

Constant ( b 0 = 22.410 b_0 = 22.410 b 0 ​ = 22.410 ) : The estimated hardness when heating time is zero minutes. (Caution: Only physically meaningful if X = 0 X = 0 X = 0 is within the operational range).

Slope ( b 1 = 3.125 b_1 = 3.125 b 1 ​ = 3.125 ) : For every additional 1.0 minute of heating time, resin hardness is predicted to increase by 3.125 Shore D units 3.125\text{ Shore D units} 3.125 Shore D units .

Standard Error of Coefficient ( SE Coef ) : Quantifies the sampling uncertainty of the parameter estimate. Smaller standard errors indicate greater precision.

T T T -Value and P P P -Value :

Tests the hypothesis: H 0 : β 1 = 0 H_0: \beta_1 = 0 H 0 ​ : β 1 ​ = 0 (no linear effect) vs H a : β 1 ≠ 0 H_a: \beta_1 \ne 0 H a ​ : β 1 ​  = 0 .

Test statistic: t = b 1 − 0 S E ( b 1 ) = 3.125 0.285 = 10.96 t = \frac{b_1 - 0}{SE(b_1)} = \frac{3.125}{0.285} = 10.96 t = SE ( b 1 ​ ) b 1 ​ − 0 ​ = 0.285 3.125 ​ = 10.96 .

Since p < 0.001 ≤ 0.05 p < 0.001 \le 0.05 p < 0.001 ≤ 0.05 , Reject H 0 H_0 H 0 ​ . Heating time has a statistically significant linear effect on hardness.

Deconstructing the Model Summary Statistics

S = 2.4512 S = 2.4512 S = 2.4512 (Standard Error of the Regression) : The estimated standard deviation of the residual errors ( M S e r r o r \sqrt{MS_{error}} M S error ​ ​ ). Represents the typical distance that observed data points fall from the regression line, measured in the units of Y Y Y .

R -Sq = 87.0 % R\text{-Sq} = 87.0\% R -Sq = 87.0% ( R 2 R^2 R 2 ) : 87.0 % 87.0\% 87.0% of the total variance in resin hardness is explained by heating time.

R -Sq(adj) = 86.2 % R\text{-Sq(adj)} = 86.2\% R -Sq(adj) = 86.2% (Adjusted R 2 R^2 R 2 ) : Modified version of R 2 R^2 R 2 that accounts for sample size and the number of predictors. In multiple regression, adding useless predictors always inflates raw R 2 R^2 R 2 , but causes R a d j 2 R^2_{adj} R a d j 2 ​ to decrease.

Deconstructing the ANOVA Table

The Analysis of Variance (ANOVA) partitions the total sum of squares into two distinct components: S S t o t a l = S S r e g r e s s i o n + S S e r r o r SS_{total} = SS_{regression} + SS_{error} S S t o t a l ​ = S S re g ress i o n ​ + S S error ​ ∑ ( y i − y ˉ ) 2 = ∑ ( y ^ i − y ˉ ) 2 + ∑ ( y i − y ^ i ) 2 \sum (y_i - \bar{y})^2 = \sum (\hat{y}_i - \bar{y})^2 + \sum (y_i - \hat{y}_i)^2 ∑ ( y i ​ − y ˉ ​ ) 2 = ∑ ( y ^ ​ i ​ − y ˉ ​ ) 2 + ∑ ( y i ​ − y ^ ​ i ​ ) 2

Source Degrees of Freedom ( D F DF D F ) Sum of Squares ( S S SS SS ) Mean Square ( M S = S S / D F MS = SS / DF MS = SS / D F ) F F F -Statistic ( M S r e g / M S e r r MS_{reg} / MS_{err} M S re g ​ / M S err ​ ) P P P -Value

Regression k = 1 k = 1 k = 1 S S r e g = 721.25 SS_{reg} = 721.25 S S re g ​ = 721.25 M S r e g = 721.25 / 1 = 721.25 MS_{reg} = 721.25 / 1 = 721.25 M S re g ​ = 721.25/1 = 721.25 F = 721.25 / 6.00 = 120.12 F = 721.25 / 6.00 = 120.12 F = 721.25/6.00 = 120.12 p < 0.001 p < 0.001 p < 0.001

Residual (Error) n − k − 1 = 18 n - k - 1 = 18 n − k − 1 = 18 S S e r r = 108.08 SS_{err} = 108.08 S S err ​ = 108.08 M S e r r = 108.08 / 18 = 6.00 MS_{err} = 108.08 / 18 = 6.00 M S err ​ = 108.08/18 = 6.00 — —

Total n − 1 = 19 n - 1 = 19 n − 1 = 19 S S t o t a l = 829.33 SS_{total} = 829.33 S S t o t a l ​ = 829.33 — — —

Exam Relationship: In simple linear regression (one predictor), the ANOVA F F F -statistic is exactly equal to the square of the slope's t t t -statistic: F = t 2 ⟹ 120.12 ≈ ( 10.96 ) 2 F = t^2 \implies 120.12 \approx (10.96)^2 F = t 2 ⟹ 120.12 ≈ ( 10.96 ) 2

Residual Analysis & The LINE Assumptions

A high R 2 R^2 R 2 and a significant p p p -value do not prove that a regression model is valid. The mathematical validity of OLS hypothesis tests, confidence intervals, and prediction bands hinges strictly upon satisfying four core assumptions, known by the acronym LINE : The Four LINE Assumptions

L — LINEARITY Relationship between X and Y is strictly linear.

I — INDEPENDENCE Residual errors are independent (no autocorrelation).

N — NORMALITY Residual errors follow a Gaussian normal distribution.

E — EQUAL VARIANCE Residual variance is constant across all values of X.

(Homoscedasticity)

To audit these assumptions, practitioners generate a diagnostic Four-in-One Residual Plot : The Four-in-One Residual Diagnostic Panel

  • 1. NORMAL PROBABILITY PLOT (Normality) 2. RESIDUALS VS. FITTED (Linearity & Equal Var)
  • Residuals ▲ Residuals ▲
  • │ █ │ █ █ █ █
  • │ █ █ │ █ █ █ █
  • │ █ █ ├───┼───┼───┼───┼───┼───┼───► Fitted (ŷ)
  • │ █ █ │ █ █ █ █
  • │ █ │ █ █ █
  • └─────────────────────────► └─────────────────────────►
  • Standard Normal Quantiles Points bounce randomly in a band
  • 3. HISTOGRAM OF RESIDUALS (Normality) 4. RESIDUALS VS. ORDER (Independence)
  • Frequency ▲ Residuals ▲
  • │ ┌───┐ │ █ █ █
  • │ ┌──┤ ├──┐ │ █ █ █ █ █ █
  • │ ┌──┤ │ │ ├──┐ ├───┼───┼───┼───┼───┼───┼───► Run Order
  • │──┴──┴──┴───┴──┴──┴──► │ █ █ █
  • Residual Bins (Centered at 0) No cyclic, drifting, or runs patterns
  • 1. Linearity ( L L L )

How to verify : Examine the Residuals versus Fitted Values plot.

Desirable Pattern : Residuals bounce randomly around the horizontal zero line within a parallel horizontal band, showing no discernable shape.

Violation Pattern : A distinct curvature (e.g., U-shape or inverted U-shape) indicates that the true relationship is non-linear. The linear model systematically overpredicts in the middle and underpredicts at the tails. Correct by adding polynomial terms (e.g., X 2 X^2 X 2 ) or transforming variables.

2. Independence ( I I I )

How to verify : Examine the Residuals versus Order (Run Order) plot.

Desirable Pattern : Random scatter across time, showing no sequences or trends.

Violation Pattern : Residuals that drift upward over time, oscillate cyclically, or exhibit long runs above or below zero indicate autocorrelation (time-dependent lurking variables such as tool wear, ambient temperature, or shift fatigue). Standard OLS standard errors become severely biased.

3. Normality ( N N N )

How to verify : Examine the Normal Probability Plot of Residuals and conduct an Anderson-Darling test.

Desirable Pattern : Residual points fall tightly along the straight diagonal reference line ( p > 0.05 p > 0.05 p > 0.05 ).

Violation Pattern : Severe S-shaped curves (indicating heavy or light tails) or severe bow shapes (indicating skewness). If non-normal, hypothesis tests ( p p p -values) lose validity, especially for small sample sizes ( n < 30 n < 30 n < 30 ).

4. Equal Variance / Homoscedasticity ( E E E )

How to verify : Examine the Residuals versus Fitted Values plot.

Desirable Pattern : Constant vertical spread of residuals across all levels of y ^ \hat{y} y ^ ​ .

Violation Pattern (Heteroscedasticity) : A distinct funnel (megaphone) shape , where residual spread widens dramatically as fitted values increase. Standard errors become invalid. Remediate using variance-stabilizing transformations (e.g., ln ⁡ ( Y ) \ln(Y) ln ( Y ) , Y \sqrt{Y} Y ​ , or Box-Cox transformation) or weighted least squares.

The Danger of Extrapolation

A critical analytical boundary in Six Sigma engineering is avoiding extrapolation :

Extrapolation is the practice of using a regression equation to predict output values ( Y Y Y ) for input values ( X X X ) that lie outside the experimental range used to calibrate the model ( x n e w < x m i n x_{new} < x_{min} x n e w ​ < x min ​ or x n e w > x m a x x_{new} > x_{max} x n e w ​ > x ma x ​ ). The Peril of Extrapolation

  • Y ▲ EXTRAPOLATION ZONE
  • │ (Unknown, Hazardous)

│ INTERPOLATION ZONE ? ? ?

│ (Valid Model Domain) ?

  • │ ┌──────────┐ ╱
  • │ │ █ █ │ ╱ (Model assumes line continues)
  • │ │ █ █ │ ╱
  • │ │█ █ │█ (Physical Reality Crashes: Phase Change)
  • │ └──────────┘ ╲
  • │ x_min x_max ╲ █
  • └──────────────────────────────────────────────┴────────────────► X
  • 0

Interpolation (Safe) : Predicting within [ x m i n , x m a x ] [x_{min}, x_{max}] [ x min ​ , x ma x ​ ] is mathematically and physically valid because empirical data supports the model throughout this domain.

Extrapolation (Dangerous) : Outside the observed window, physical systems frequently experience regime changes: chemical catalysts saturate, metal structures hit yield points, fluids boil or freeze, and electronic components enter thermal runaway. A line that fit brilliantly between 50 ∘ C 50^\circ\text{C} 5 0 ∘ C and 100 ∘ C 100^\circ\text{C} 10 0 ∘ C may fail completely at 150 ∘ C 150^\circ\text{C} 15 0 ∘ C .

Step-by-Step Worked Calculation Example

Problem: A packaging line engineer monitors sealing temperature ( X X X , in ∘ C ^\circ\text{C} ∘ C ) and seal rupture force ( Y Y Y , in Newtons). A calibration study yields the following summary statistics for n = 10 n = 10 n = 10 test pouches:

x ˉ = 150.0 ∘ C \bar{x} = 150.0^\circ\text{C} x ˉ = 150. 0 ∘ C , s x = 10.0 ∘ C s_x = 10.0^\circ\text{C} s x ​ = 10. 0 ∘ C , S S x x = 900.0 SS_{xx} = 900.0 S S xx ​ = 900.0

y ˉ = 85.0 N \bar{y} = 85.0\text{ N} y ˉ ​ = 85.0 N , s y = 12.0 N s_y = 12.0\text{ N} s y ​ = 12.0 N , S S y y = 1296.0 SS_{yy} = 1296.0 S S yy ​ = 1296.0

S S x y = 972.0 SS_{xy} = 972.0 S S x y ​ = 972.0

Calculate the slope b 1 b_1 b 1 ​ and intercept b 0 b_0 b 0 ​ .

State the regression equation.

Predict the rupture force for a temperature of 160.0 ∘ C 160.0^\circ\text{C} 160. 0 ∘ C .

If a test pouch sealed at 160.0 ∘ C 160.0^\circ\text{C} 160. 0 ∘ C exhibits an actual rupture force of 98.0 N 98.0\text{ N} 98.0 N , compute the residual.

Step 1: Calculate Slope ( b 1 b_1 b 1 ​ ) and Intercept ( b 0 b_0 b 0 ​ ) b 1 = S S x y S S x x = 972.0 900.0 = 1.08 N / ∘ C b_1 = \frac{SS_{xy}}{SS_{xx}} = \frac{972.0}{900.0} = 1.08\text{ N}/^\circ\text{C} b 1 ​ = S S xx ​ S S x y ​ ​ = 900.0 972.0 ​ = 1.08 N / ∘ C b 0 = y ˉ − b 1 x ˉ = 85.0 − ( 1.08 × 150.0 ) = 85.0 − 162.0 = − 77.0 N b_0 = \bar{y} - b_1 \bar{x} = 85.0 - (1.08 \times 150.0) = 85.0 - 162.0 = -77.0\text{ N} b 0 ​ = y ˉ ​ − b 1 ​ x ˉ = 85.0 − ( 1.08 × 150.0 ) = 85.0 − 162.0 = − 77.0 N

Step 2: Formulate the Prediction Equation y ^ = − 77.0 + 1.08 x \hat{y} = -77.0 + 1.08x y ^ ​ = − 77.0 + 1.08 x

Interpretation: For each 1.0 ∘ C 1.0^\circ\text{C} 1. 0 ∘ C increase in sealing temperature, seal rupture force is predicted to increase by 1.08 N 1.08\text{ N} 1.08 N . (Note: The intercept − 77.0 N -77.0\text{ N} − 77.0 N has no physical meaning because a temperature of 0 ∘ C 0^\circ\text{C} 0 ∘ C is far outside the operating window).

Step 3: Predict Rupture Force at x = 160.0 ∘ C x = 160.0^\circ\text{C} x = 160. 0 ∘ C y ^ = − 77.0 + 1.08 ( 160.0 ) = − 77.0 + 172.8 = 95.8 N \hat{y} = -77.0 + 1.08(160.0) = -77.0 + 172.8 = 95.8\text{ N} y ^ ​ = − 77.0 + 1.08 ( 160.0 ) = − 77.0 + 172.8 = 95.8 N

Step 4: Calculate the Residual ( e e e ) Residual e = y a c t u a l − y ^ p r e d i c t e d = 98.0 − 95.8 = + 2.2 N \text{Residual } e = y_{actual} - \hat{y}_{predicted} = 98.0 - 95.8 = +2.2\text{ N} Residual e = y a c t u a l ​ − y ^ ​ p re d i c t e d ​ = 98.0 − 95.8 = + 2.2 N

The actual seal was 2.2 N 2.2\text{ N} 2.2 N stronger than predicted by the regression line.

Critical Exam Traps to Avoid

Trap 1: Extrapolating Beyond the Observed Range of X X X — Questions will present a model built on temperatures between 100 ∘ C 100^\circ\text{C} 10 0 ∘ C and 200 ∘ C 200^\circ\text{C} 20 0 ∘ C and ask for a prediction at 450 ∘ C 450^\circ\text{C} 45 0 ∘ C . The correct answer is to flag extrapolation as statistically hazardous.

Trap 2: Declaring a Model Valid Based on High R 2 R^2 R 2 Alone — A regression model can boast R 2 = 95 % R^2 = 95\% R 2 = 95% while suffering from severe heteroscedasticity, strong autocorrelation, or severe non-linearity. High R 2 R^2 R 2 never excuses omitting residual diagnostic plots.

Trap 3: Misinterpreting the Intercept ( b 0 b_0 b 0 ​ ) — In many physical processes, b 0 b_0 b 0 ​ has no realistic operational interpretation if X = 0 X = 0 X = 0 is physically impossible (e.g., zero human height, zero absolute pressure, or zero absolute temperature in Celsius).

Trap 4: Inverting the Residual Formula — Remember that residual is always Actual minus Predicted ( e = y − y ^ e = y - \hat{y} e = y − y ^ ​ ). Inverting it as y ^ − y \hat{y} - y y ^ ​ − y will reverse all diagnostic signs.

Loading diagram... Simple Linear Regression Workflow & Residual Validation Protocol