6.5 Data Collection Plans and Common Pitfalls

ASQ Certified Six Sigma Black Belt CSSBB

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & CSSBB Exam Strategy

1.1 CSSBB Exam Format, Scoring, and Testing Windows 1.2 Eligibility, Project Affidavits, Application, and Recertification 1.3 Open-Book Strategy, Reference Materials, and a 16-Week Study Plan

2 Chapter 2: Organization-Wide Planning and Deployment

2.1 Fundamentals of Six Sigma and Lean Methodologies 2.2 Selecting the Right Methodology: DMAIC, DFSS, and Lean 2.3 Business Systems, Processes, and Stakeholder Relationships 2.4 Strategic Planning and Deployment of Six Sigma Initiatives 2.5 Six Sigma Roles and Responsibilities 2.6 Organizational Barriers and Common Causes of Six Sigma Failure 2.7 Change Management: Stakeholder Analysis, Readiness, and Communication

3 Chapter 3: Organizational Process Management and Measures

3.1 Impact of Six Sigma Projects on Customers, Suppliers, and Stakeholders 3.2 Benchmarking Types and Performance Goal Setting 3.3 Performance Measures: Balanced Scorecard, KPIs, KBIs, OKRs, and Line of Sight 3.4 Leading and Lagging Indicators and Performance Dashboards 3.5 Financial Measures: Revenue Growth, Margin, NPV, ROI, and Cost-Benefit Analysis 3.6 Cost of Poor Quality and Hard versus Soft Savings

4 Chapter 4: Team Management

4.1 Team Types and Constraints 4.2 Team Roles, Responsibilities, and Member Selection 4.3 Team Success Factors and Ground Rules 4.4 Motivational Techniques and Overcoming Demotivators 4.5 Stages of Team Development and Conflict Resolution 4.6 Team Communication Plans 4.7 Team Leadership Models: Direct, Coach, Support, and Delegate 4.8 Group Behaviors, Conflict Resolution, and Meeting Management 4.9 Facilitation Techniques and Team Decision-Making Methods 4.10 Team Training: Needs Assessment, Delivery, and Evaluation

5 Chapter 5: Define Phase

5.1 Customer Identification and Segmentation 5.2 Customer Data Collection: Methods, Validity, and Reliability 5.3 Customer Requirements: CTX, CTQ Trees, and Operational Definitions 5.4 The Kano Model and Quality Function Deployment 5.5 SIPOC and Process Mapping 5.6 Business Case Justification 5.7 The Project Charter and the Problem Statement 5.8 Project Scope and Boundaries 5.9 SMART Goals and Project Performance Measurements 5.10 Project Charter Reviews and Toll-Gate Governance 5.11 Project Management Tools: Gantt Charts, Toll-Gate Reviews, WBS, and RACI 5.12 Analytical Tools Used Throughout the DMAIC Cycle

6 Chapter 6: Measure Phase

6.1 Process Flow Metrics and the Hidden Factory 6.2 Process Analysis Tools and Value Stream Mapping 6.3 Types of Data and Measurement Scales 6.4 Sampling Concepts, Methods, and Sample Size for Estimation 6.5 Data Collection Plans and Common Pitfalls 6.6 Gage R&R for Variable Measurement Systems 6.7 Attribute Agreement Analysis and Audit MSA 6.8 Measurement Systems Across the Organization, and Metrology 6.9 Population Parameters, Sample Statistics, and Valid Statistical Conclusions 6.10 Descriptive Statistics: Central Tendency, Dispersion, and Shape 6.11 Graphical Methods for Exploratory Data Analysis 6.12 Probability Concepts, Common Distributions, and the Central Limit Theorem 6.13 Additional Distributions: Hypergeometric, Exponential, Lognormal, Weibull, and Bivariate 6.14 Process Capability and Performance Indices 6.15 Capability Studies, Attributes Capability, Yield Metrics, and Long-Term Capability

7 Chapter 7: Analyze Phase

7.1 The Correlation Coefficient, Its Confidence Interval, and Causation 7.2 Linear Regression and Residual Analysis 7.3 Multivariate Tools: Factor Analysis, Discriminant Analysis, and MANOVA 7.4 Hypothesis Testing Terminology, Errors, and Single-Sample Tests 7.5 Statistical Versus Practical Significance 7.6 Sample Size for Hypothesis Tests 7.7 Point Estimates, Confidence, Prediction, and Tolerance Intervals 7.8 Tests for Means, Variances, and Proportions 7.9 Analysis of Variance 7.10 Chi-Square Goodness-of-Fit and Tests of Independence 7.11 Types of Risk and Failure Mode and Effects Analysis 7.12 Gap Analysis and Root Cause Analysis 7.13 Waste Analysis: The Seven Classic Wastes and Resource Under-Utilization

8 Chapter 8: Improve Phase

8.1 Design of Experiments: Objectives, Terminology, and Effects 8.2 DOE Design Principles: Randomization, Blocking, Replication, and Resolution 8.3 Planning and Evaluating Designed Experiments 8.4 One-Factor Experiments: Completely Randomized, Randomized Block, and Latin Square Designs 8.5 Full Factorial Experiments and Design Selection 8.6 Two-Level Fractional Factorial Experiments and Confounding 8.7 Response Surface Methodology and Optimization 8.8 Waste Elimination: Pull Systems, Kanban, 5S, and Poka-Yoke 8.9 Cycle-Time Reduction: Continuous Flow, SMED, and Heijunka 8.10 Kaizen, Kaizen Blitz, Theory of Constraints, and OEE 8.11 Implementation Planning, Piloting, and Solution Selection

9 Chapter 9: Control Phase

9.1 SPC Objectives, Variable Selection, and Rational Subgrouping 9.2 Control Charts for Variables Data 9.3 Control Charts for Attributes Data 9.4 Short-Run SPC and Moving Average Charts 9.5 Control Chart Analysis: Common Cause, Special Cause, and Run Rules 9.6 Total Productive Maintenance and Visual Controls 9.7 Measurement System Reanalysis 9.8 Control Plan Development and Process Owner Handoff 9.9 Lessons Learned, Documentation, and Training for Sustainment 9.10 Ongoing Evaluation of the Improved Process

10 Chapter 10: Design for Six Sigma (DFSS)

10.1 DFSS Frameworks: DMADV, DMADOV, and Design Transfer 10.2 Design for X (DFX) 10.3 Robust Design, Parameter Design, and Verification 10.4 Tolerance Design and Statistical Tolerancing

6.5 Data Collection Plans and Common Pitfalls

Key Takeaways Explain This Topic AI

A data collection plan specifies what, why, how, who, when, where, how many, and how the data will be recorded and stratified.

The three pitfalls ASQ names are undefined metrics, collectors untrained in the tools or unaware of how the data will be used, and unchecked seasonality effects.

Stratification factors must be captured at the moment of collection, because they cannot be reconstructed afterwards.

Check sheets are the standard structured collection instrument and should be designed so that recording is faster than not recording.

Data normalization converts raw counts into comparable rates, without which comparisons across units of different size are invalid.

Last updated: August 2026

The plan

A data collection plan is a table, one row per measure, answering eight questions.

Field Content Example

What The measure and its operational definition Fill weight in grams, measured at the check-weigher, at the moment of seal

Why Which project question it answers Baseline capability against 500 g +/- 8 g
How Instrument, method, resolution Check-weigher #3, 0.1 g resolution, MSA completed 12 Aug
Who Named role, trained and certified Line QA technician, certified on the work instruction
When Frequency and timing 5 consecutive units every 30 minutes
Where Location and stratification Lines 1-3, all shifts, both resin lots
How many Sample size with its justification 25 subgroups of 5 = 125 units, for control chart establishment
Recording Form, field names, destination, retention Check sheet CS-114, keyed to the MES within one shift

Two fields are habitually skipped and both are costly. Why prevents collection of data nobody uses. Where , in the sense of stratification factors, cannot be added retrospectively.

Data integrity and accuracy

Distinguish three things that are frequently conflated:

Accuracy -- how close the measurement is to the true value. Addressed by calibration and by bias

studies in MSA.

Precision -- how repeatable the measurement is. Addressed by gage R&R.

Integrity -- whether the record faithfully represents what happened. Addressed by collection

design, training, and traceability.

Integrity failures are the ones a statistical analysis cannot detect. Transcription errors, entries
made at the end of shift from memory, defaults left unchanged, unit mix-ups, and duplicated records all

produce data that looks well-behaved and is wrong.

Practical safeguards:

Record at the point and moment of occurrence, never retrospectively.

Use fixed value lists rather than free text for categorical fields.

Include a unique identifier per record so duplicates are detectable.

Include the collector, the instrument ID, and the timestamp on every record.

Range-check and plausibility-check on entry; a fill weight of 5,000 g should be rejected at the

keyboard.

Reconcile counts: units measured should tie to units produced.

The three pitfalls named by ASQ
1. Metrics not defined
If the metric is not operationally defined, different collectors record different things and the
variation you measure is partly theirs. Two collectors working independently from the definition alone
must classify the same items identically -- and attribute agreement analysis is the formal test of
whether they do. Where a definition proves ambiguous, fix the definition and re-collect rather than

adjudicating the disputed records.

2. Collectors untrained and unaware of how data will be used
The Body of Knowledge is specific that collectors must be trained in the tools and understand how

the data will be used. The second half is the part usually skipped, and it matters for two reasons.

People who know the purpose catch anomalies -- "this reading looks wrong, the gauge was just knocked" --
that a mechanical recorder would enter unchallenged. And people who do not know the purpose invent
one, which is where fear-driven distortion begins. State explicitly that the study examines the

process and not the individual.

3. Seasonality not checked
Seasonality here means any systematic time-related pattern: hour of day, day of week, shift, week of

month, month of year, or campaign and lot cycles.

Two distinct failures follow from ignoring it. A baseline collected during an unrepresentative window
gives a wrong baseline -- a call centre measured in the first week of January is not the same process
as in the third week of June. And a before-and-after comparison that spans a seasonal change credits
the season to the project. The remedies are to collect across at least one full cycle of the dominant
period, to stratify by time factors, and where a full cycle is impossible, to compare like periods

year over year rather than consecutive periods.

Stratification factors

Capture them at collection time. The standard candidates:

Factor Why it matters
Machine, line, or station Equipment differences are a leading variation source
Operator or team Method differences between people
Shift Staffing, supervision, and support differ by shift
Time of day / day of week Warm-up, fatigue, demand pattern
Material lot or supplier Input variation
Product, SKU, or customer segment Different requirements and routings
Location or site Environment, local practice
The rule: if you might later want to compare two groups, you must record which group each observation
belongs to at the time you record it. Reconstructing "which shift produced this unit" from timestamps

three months later is unreliable and sometimes impossible.

Check sheets
A check sheet is a pre-formatted form that structures recording at the point of collection. Common
forms: the defect-type tally , the defect-location diagram (marks placed on a picture of the
product), the defect cause sheet crossing cause categories with time, and the tally

distribution sheet that builds a histogram as data arrives.

Design rules:

Make recording faster than not recording. A tick beats a written number; a written number beats a

sentence.

Pre-print all categories, including "other" with a free-text prompt so novel modes are captured.

Include the stratification fields as pre-printed columns rather than relying on memory.

Pilot the sheet for one shift before full deployment. Every check sheet has an error in its first

version.

Data normalization
Raw counts are not comparable across units of different size, volume, or exposure. Normalization

converts them into rates: Defect rate = Defects Opportunities , D P U = Defects Units , Rate = Events Exposure \text{Defect rate} = \frac{\text{Defects}}{\text{Opportunities}}, \qquad DPU = \frac{\text{Defects}}{\text{Units}}, \qquad \text{Rate} = \frac{\text{Events}}{\text{Exposure}} Defect rate = Opportunities Defects , D P U = Units Defects , Rate = Exposure Events

Plant A reporting 340 defects and plant B reporting 95 says nothing until you know plant A produced
120,000 units and plant B produced 11,000: the rates are 2.8 and 8.6 per thousand, and the smaller
plant is three times worse. Choose the denominator that represents genuine exposure -- units, hours,
opportunities, patient-days, transactions -- and use the same denominator on both sides of every

comparison.

Before you start collecting

A short pre-flight checklist:

Is the measurement system validated (MSA complete and passed)?

Is every metric operationally defined and tested for agreement between collectors?

Are the collectors trained, and do they know how the data will be used?

Are all stratification factors on the form?

Does the collection window cover at least one full cycle of the dominant time pattern?

Is the sample size justified by a calculation rather than by convenience?

Has the form been piloted for one shift?

Is there a plan for missing and out-of-range values?