Study guide
8.3 Planning and Evaluating Designed Experiments
ASQ Certified Six Sigma Black Belt CSSBB
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Introduction & CSSBB Exam Strategy
1.1 CSSBB Exam Format, Scoring, and Testing Windows 1.2 Eligibility, Project Affidavits, Application, and Recertification 1.3 Open-Book Strategy, Reference Materials, and a 16-Week Study Plan
2 Chapter 2: Organization-Wide Planning and Deployment
2.1 Fundamentals of Six Sigma and Lean Methodologies 2.2 Selecting the Right Methodology: DMAIC, DFSS, and Lean 2.3 Business Systems, Processes, and Stakeholder Relationships 2.4 Strategic Planning and Deployment of Six Sigma Initiatives 2.5 Six Sigma Roles and Responsibilities 2.6 Organizational Barriers and Common Causes of Six Sigma Failure 2.7 Change Management: Stakeholder Analysis, Readiness, and Communication
3 Chapter 3: Organizational Process Management and Measures
3.1 Impact of Six Sigma Projects on Customers, Suppliers, and Stakeholders 3.2 Benchmarking Types and Performance Goal Setting 3.3 Performance Measures: Balanced Scorecard, KPIs, KBIs, OKRs, and Line of Sight 3.4 Leading and Lagging Indicators and Performance Dashboards 3.5 Financial Measures: Revenue Growth, Margin, NPV, ROI, and Cost-Benefit Analysis 3.6 Cost of Poor Quality and Hard versus Soft Savings
4 Chapter 4: Team Management
4.1 Team Types and Constraints 4.2 Team Roles, Responsibilities, and Member Selection 4.3 Team Success Factors and Ground Rules 4.4 Motivational Techniques and Overcoming Demotivators 4.5 Stages of Team Development and Conflict Resolution 4.6 Team Communication Plans 4.7 Team Leadership Models: Direct, Coach, Support, and Delegate 4.8 Group Behaviors, Conflict Resolution, and Meeting Management 4.9 Facilitation Techniques and Team Decision-Making Methods 4.10 Team Training: Needs Assessment, Delivery, and Evaluation
5 Chapter 5: Define Phase
5.1 Customer Identification and Segmentation 5.2 Customer Data Collection: Methods, Validity, and Reliability 5.3 Customer Requirements: CTX, CTQ Trees, and Operational Definitions 5.4 The Kano Model and Quality Function Deployment 5.5 SIPOC and Process Mapping 5.6 Business Case Justification 5.7 The Project Charter and the Problem Statement 5.8 Project Scope and Boundaries 5.9 SMART Goals and Project Performance Measurements 5.10 Project Charter Reviews and Toll-Gate Governance 5.11 Project Management Tools: Gantt Charts, Toll-Gate Reviews, WBS, and RACI 5.12 Analytical Tools Used Throughout the DMAIC Cycle
6 Chapter 6: Measure Phase
6.1 Process Flow Metrics and the Hidden Factory 6.2 Process Analysis Tools and Value Stream Mapping 6.3 Types of Data and Measurement Scales 6.4 Sampling Concepts, Methods, and Sample Size for Estimation 6.5 Data Collection Plans and Common Pitfalls 6.6 Gage R&R for Variable Measurement Systems 6.7 Attribute Agreement Analysis and Audit MSA 6.8 Measurement Systems Across the Organization, and Metrology 6.9 Population Parameters, Sample Statistics, and Valid Statistical Conclusions 6.10 Descriptive Statistics: Central Tendency, Dispersion, and Shape 6.11 Graphical Methods for Exploratory Data Analysis 6.12 Probability Concepts, Common Distributions, and the Central Limit Theorem 6.13 Additional Distributions: Hypergeometric, Exponential, Lognormal, Weibull, and Bivariate 6.14 Process Capability and Performance Indices 6.15 Capability Studies, Attributes Capability, Yield Metrics, and Long-Term Capability
7 Chapter 7: Analyze Phase
7.1 The Correlation Coefficient, Its Confidence Interval, and Causation 7.2 Linear Regression and Residual Analysis 7.3 Multivariate Tools: Factor Analysis, Discriminant Analysis, and MANOVA 7.4 Hypothesis Testing Terminology, Errors, and Single-Sample Tests 7.5 Statistical Versus Practical Significance 7.6 Sample Size for Hypothesis Tests 7.7 Point Estimates, Confidence, Prediction, and Tolerance Intervals 7.8 Tests for Means, Variances, and Proportions 7.9 Analysis of Variance 7.10 Chi-Square Goodness-of-Fit and Tests of Independence 7.11 Types of Risk and Failure Mode and Effects Analysis 7.12 Gap Analysis and Root Cause Analysis 7.13 Waste Analysis: The Seven Classic Wastes and Resource Under-Utilization
8 Chapter 8: Improve Phase
8.1 Design of Experiments: Objectives, Terminology, and Effects 8.2 DOE Design Principles: Randomization, Blocking, Replication, and Resolution 8.3 Planning and Evaluating Designed Experiments 8.4 One-Factor Experiments: Completely Randomized, Randomized Block, and Latin Square Designs 8.5 Full Factorial Experiments and Design Selection 8.6 Two-Level Fractional Factorial Experiments and Confounding 8.7 Response Surface Methodology and Optimization 8.8 Waste Elimination: Pull Systems, Kanban, 5S, and Poka-Yoke 8.9 Cycle-Time Reduction: Continuous Flow, SMED, and Heijunka 8.10 Kaizen, Kaizen Blitz, Theory of Constraints, and OEE 8.11 Implementation Planning, Piloting, and Solution Selection
9 Chapter 9: Control Phase
9.1 SPC Objectives, Variable Selection, and Rational Subgrouping 9.2 Control Charts for Variables Data 9.3 Control Charts for Attributes Data 9.4 Short-Run SPC and Moving Average Charts 9.5 Control Chart Analysis: Common Cause, Special Cause, and Run Rules 9.6 Total Productive Maintenance and Visual Controls 9.7 Measurement System Reanalysis 9.8 Control Plan Development and Process Owner Handoff 9.9 Lessons Learned, Documentation, and Training for Sustainment 9.10 Ongoing Evaluation of the Improved Process
10 Chapter 10: Design for Six Sigma (DFSS)
10.1 DFSS Frameworks: DMADV, DMADOV, and Design Transfer 10.2 Design for X (DFX) 10.3 Robust Design, Parameter Design, and Verification 10.4 Tolerance Design and Statistical Tolerancing
8.3 Planning and Evaluating Designed Experiments
Key Takeaways Explain This Topic AI
The planning sequence is objective, response, factors and levels, design selection, run plan, execution, analysis, and confirmation.
Factor ranges must be wide enough to produce a detectable effect but inside safe and feasible operating limits.
Screening designs identify the vital few factors; characterization designs quantify effects and interactions; optimization designs locate the best settings.
A confirmation run at the predicted optimum is mandatory, because a model is a prediction until it is verified.
The measurement system must be validated before the experiment, because measurement variation enters the error term and masks real effects.
Last updated: August 2026
The planning sequence
- Step Question Failure if skipped
- 1. Objective Screening, characterization, or optimization? Wrong design chosen
- 2. Response What is measured, how, and is the gauge validated? Measurement noise masks real effects
3. Factors and levels Which factors, at what settings, held how constant? Missing factor becomes a lurking variable
- 4. Design Which design, at what resolution, with what replication? Confounded results that cannot be resolved
- 5. Run plan Randomized order, blocking, who runs it, over what period Systematic bias
6. Execution Actual settings recorded; deviations logged Analysis of the design you intended, not the one you ran
- 7. Analysis Effects, ANOVA, residual diagnostics, reduced model Overfitted or invalid model
- 8. Confirmation Runs at the predicted optimum Prediction never verified
- Step 1: The objective determines the design
Objective Typical factor count Design family Runs
Screening -- find the vital few from many candidates 5-15 Resolution III or IV fractional factorial; Plackett-Burman 8-32
Characterization -- quantify main effects and interactions 2-5 Full factorial or Resolution V fraction, with centre points 16-64
Optimization -- find the settings that optimize the response 2-4 Response surface: central composite, Box-Behnken 15-30
Robustness -- find settings insensitive to noise 2-6 controllable plus noise Robust parameter design, inner/outer array Varies
The sequential logic matters: screen first, characterize the survivors, then optimize. Attempting to
optimize eight factors in one experiment is expensive and usually finds that five of them do not matter.
Step 2: The response
The response must be:
Continuous where possible. Attribute responses need far more runs for the same sensitivity.
- Measured by a validated system. Observed variance is process variance plus measurement variance;
- a gauge consuming 30% of the variation can hide a real effect entirely. Complete the gage R&R before
the experiment, not after a disappointing result.
Sensitive to the factors. A response measured with resolution coarser than the effect size cannot
detect it.
Related to the CTQ. Optimizing a convenient surrogate that the customer does not experience is a
common waste.
- Multiple responses are normal, and they frequently conflict -- higher throughput against lower defect
- rate. Plan for that with desirability functions or overlaid contour plots rather than optimizing one
and hoping.
- Step 3: Factors, levels, and ranges
- Sources for the candidate factor list: the process map, the cause-and-effect diagram, the PFMEA, the
Analyze-phase findings, and operator knowledge. Then classify each candidate:
- Class Treatment in the experiment
- Controllable and of interest Include as an experimental factor
- Controllable but not of interest Hold constant and record the value
- Uncontrollable but measurable Record as a covariate; consider blocking
- Uncontrollable and unmeasurable Randomize against it
- Setting the levels
Range selection is the judgment that most affects whether the experiment succeeds.
- Too narrow and the effect is smaller than experimental error, so a real factor is declared
- inactive. This is by far the more common error, because engineers instinctively stay near the current
operating point.
Too wide and you risk scrap, safety issues, equipment damage, or a region where the
response is no longer approximately linear between the two levels.
- Practical guidance: set the levels near the edges of the safe operating window , not near the current
- setting. A useful check is whether the two levels would be expected to produce a visible difference to
someone who knows the process; if not, widen them.
Two-level factors assume approximate linearity across the range. Centre points test that assumption:
if the average of the centre points differs significantly from the average of the factorial points,
curvature is present and a response surface design is needed.
- Step 4-6: Design, run plan, execution
- Once objective, response, and factors are set, the design follows almost mechanically from the run
budget and the required resolution. The run plan then adds:
Randomized run order , generated and printed before starting.
Blocking for known nuisance sources -- day, lot, operator, machine.
A data sheet with the actual settings achieved, not just the intended ones, plus a notes column.
A defined procedure for what to do when a run cannot be made: never silently substitute another
setting.
- During execution, record what actually happened . Analyzing the design you intended when the operator
- could not reach 220 C and ran 210 C instead produces a wrong model. Deviations are recoverable if
recorded and unrecoverable if not.
Step 7: Analysis
Plot the raw data first -- main effects plots and interaction plots.
Estimate the effects; use a normal probability plot of effects or a Pareto of effects to identify
the active ones. Inactive effects fall on a straight line through zero.
Build the ANOVA and check significance.
Check the residuals : normal probability plot, residuals versus fits, residuals versus run order.
The last one detects time-related drift that randomization converted into error.
Reduce the model to the significant terms, respecting hierarchy -- if an interaction is retained,
keep both parent main effects even if individually non-significant.
Report R 2 R^2 R 2 and adjusted R 2 R^2 R 2 . A large gap between them indicates overfitting.
- Step 8: Confirmation
- The confirmation run is not optional. Run several replicates at the predicted optimum and compare the
observed response with the model's prediction interval.
- Outcome Interpretation
- Observed inside the prediction interval Model validated; proceed to pilot and implementation
Observed outside the interval Model is wrong: a missing factor, extrapolation beyond the design space, or an unstable process
A model that predicts well inside the experimental region can still fail outside it. Never extrapolate:
the model is only supported over the ranges actually run.
Common planning failures
- Failure Consequence Prevention
- Measurement system not validated Real effects hidden in noise MSA before the experiment
- Ranges too narrow Important factors declared inactive Set levels near the safe operating limits
Important factor omitted and not held constant Becomes a lurking variable Structured factor identification from the PFMEA and process map
- Run order not randomized Time-related drift confounded with factors Generate and follow a randomized order
- No replication or centre points No valid error estimate Add replicates or centre points
Process unstable during the experiment Error term inflated; nothing is significant Establish statistical control first