19.2 Learning Outcomes and Evaluation Data

BCSP CIT Certified Instructional Trainer Study Guide 2026

Track your progress as you read

Finished sections get a check mark.

1 Credential and Training Standards

1.1 BCSP CIT Credential Scope, Exam Logistics & Recertification 1.2 ANSI/ASSP Z490 Criteria for Accepted EHS Training Practices

2 Adult Learning and Learning Access

2.1 Foundations of Andragogy: Knowles' Adult Learning Assumptions in Safety 2.2 Learner Literacy, ESL Accommodations & Plain Language Compliance 2.3 Opening a Course: Administrative Items and Learning Expectations

3 Communication and Facilitation

3.1 Verbal & Nonverbal Facilitation Techniques in Safety Training 3.2 Active Listening, Socratic Questioning & Teach-Back Verification 3.3 Feedback Frameworks, Corrective Coaching & Psychological Safety

4 Conflict and Instructional Technology

4.1 Managing Challenging Classroom Dynamics, Resistance & Conflict 4.2 Instructional Media, Presentation Tools & Technology Contingencies

5 Performance Needs and Existing Training

5.1 Performance Gaps and Training Necessity 5.2 Evaluating Existing Training and Interventions

6 Organizational, Task and Hazard Analysis

6.1 Organizational and Job Task Analysis 6.2 Hazard Analysis, KSAs and Task Evidence

7 Audience, Baseline and Prerequisites

7.1 Defining the Audience and Its Learning Needs 7.2 Baseline Knowledge and Prior-Learning Credit 7.3 Verifying Prerequisites and Readiness for Training

8 Applicable Training Requirements

8.1 OSHA Training Frameworks, General Duty Clause & Consensus Standards 8.2 Mandated Safety Programs: Lockout/Tagout, Confined Space & Fall Protection 8.3 Chemical & Biological Hazards: HazCom, Bloodborne & Respiratory Protection

9 Systematic and Iterative Design

9.1 ADDIE and the Systematic Approach to Training 9.2 SAM, Agile Development and Prototypes

10 Objectives and Learning Outcomes

10.1 Writing Observable ABCD Objectives 10.2 SMART, Objective Hierarchies and Stakeholder Consensus

11 Learning Domains and Practice

11.1 Cognitive Learning and Revised Bloom Taxonomy 11.2 Psychomotor, Affective and Constructive Alignment 11.3 Experiential Learning Cycles (Kolb), Adult Discovery & Instructional Scaffolding

12 Delivery Choices and Program Economics

12.1 Modality Matrix: Classroom ILT, E-Learning, Microlearning, Hands-On & OJT 12.2 Cost Classification and Worked Budgets 12.3 Resource Feasibility and Cost-Benefit Decisions

13 Developing and Validating Course Components

13.1 Lesson Plan Architecture: Components of Comprehensive Trainer & Participant Guides 13.2 Job Aids and Performance Support 13.3 SME Selection, Credible Sources and Validation

14 Development Planning and External Providers

14.1 Implementing the Instructional Development Plan 14.2 Deciding When and How to Outsource Development

15 Pilots and Recoverable Records

15.1 Representative Pilots and Course Revision 15.2 Record Retention, Privacy and Recovery

16 Implementation Environments and Logistics

16.1 Physical Classroom Design, Seating Arrangements, Ergonomics & Environmental Controls 16.2 Field, Shop & Hands-On Training: Safety Briefings, PPE Verification & Live Demonstration Safety 16.3 Virtual Instructor-Led Training (VILT), Technical Contingency Planning & Remote Engagement 16.4 Scheduling, Logistics and Course Time Management

17 Assessment and Remediation

17.1 Formative and Summative Assessment Tools 17.2 Assessment Criteria, Alignment and Remediation

18 Items, Performance and Standards

18.1 Writing Clear Items and Plausible Distractors 18.2 Item Analysis and Assessment Data 18.3 Performance Checklists and Evaluator Calibration 18.4 Standard Setting and the Angoff Calculation

19 Evaluation Models, Surveys and Learning Data

19.1 Evaluation Models and Feedback Surveys 19.2 Learning Outcomes and Evaluation Data

20 Transfer and Behavior Evidence

20.1 Transfer of Learning and Work Conditions 20.2 Behavior Observations and Change Reporting

21 Results, ROI and Improvement Reports

21.1 Organizational Results and Causal Limits 21.2 ROI, Course Improvement and Training Reports

19.2 Learning Outcomes and Evaluation Data

Key Takeaways Explain This Topic AI

Pre/post comparisons need comparable outcomes, populations and scoring conditions.

Normalized gain is descriptive and does not prove causality or individual mastery.

A pre-test score of 100 makes the normalized-gain denominator zero.

Last updated: October 2026

Level 2: Learning — Objective Assessment and Pre/Post Testing

Level 2 concerns the intended learning. Use the knowledge, reasoning or physical-performance evidence appropriate to the objectives. Do not require every course to combine every instrument, and do not infer physical skill from a written test alone.

Pre-Test and Post-Test Methodology

Administering identical or psychometrically parallel assessments before and after training provides empirical baseline comparison. However, raw percentage point increases ( Post − Pre \text{Post} - \text{Pre} Post − Pre ) distort true instructional effectiveness because students with high baseline scores have limited room for numerical improvement—a psychometric artifact known as the ceiling effect . Conversely, very difficult tests can generate floor effects , masking foundational gains.

Normalized gain as a descriptive calculation

A normalized gain expresses observed score improvement as a fraction of the available improvement from the pre-test score. It can be useful descriptively but does not neutralize all baseline differences or prove training caused the gain. It is undefined for a pre-test score of 100 and can be negative. g = % Post-test − % Pre-test 100 % − % Pre-test g = \frac{\% \text{Post-test} - \% \text{Pre-test}}{100\% - \% \text{Pre-test}} g = 100% − % Pre-test % Post-test − % Pre-test ​

Do not apply universal low, medium and high gain categories to every occupational course. Interpret the result with test quality, sample, baseline, task requirements and other evidence.

Worked Calculation Example

A cohort of 24 chemical process technicians completes an OSHA Process Safety Management (PSM) training module.

Baseline cohort pre-test average: 55 % 55\% 55%

Post-instruction cohort post-test average: 82 % 82\% 82% g = 82 − 55 100 − 55 = 27 45 = 0.60 g = \frac{82 - 55}{100 - 55} = \frac{27}{45} = 0.60 g = 100 − 55 82 − 55 ​ = 45 27 ​ = 0.60

For these hypothetical cohort averages, the observed normalized gain is 0.60. This does not mean every learner gained equally or that workplace behavior improved. Review individual critical competencies and the design of the comparison.

Controlling Threats to Internal Validity

When conducting pre/post testing, trainers must recognize and control threats to internal validity:

Pre-Test Sensitization (Testing Effect) : Taking a pre-test alerts learners to key exam concepts, prompting them to pay disproportionate attention to those topics during class. Control : Utilize item-pool randomization or counterbalanced split-half testing (Group A receives Form 1 pre and Form 2 post; Group B receives Form 2 pre and Form 1 post).

Instrumentation Drift : Changing test difficulty, scoring rubrics, or proctor leniency between administrations. Control : Use standardized, validated rubrics and psychometrically parallel items.

Statistical Regression Toward the Mean : Extreme low scorers naturally tend to score closer to the average on re-testing simply due to random variance. Control : Include an uninstructed control group or calculate normalized gains across matched pairs.

Note

A physical performance objective needs appropriate performance evidence. A multiple-choice knowledge gain or positive reaction alone cannot establish it.

Collecting data that supports a comparison

Specify the learner population, instruments, dates and conditions before collecting results. A pre/post comparison needs comparable evidence of the intended outcome. Identical questions can introduce familiarity; alternate forms can differ in difficulty. Neither choice automatically solves the problem. Document the design and its limitations, and use additional performance evidence where the outcome requires it.

Pair learner records appropriately when examining individual change. An average pre-test from one group and average post-test from a partly different group can reflect composition rather than learning. Report missing data and interruptions. Preserve privacy with suitable identifiers and avoid displaying identifiable scores in a class-wide report.

• Data check Reason
• Same outcome and scoring criteria Comparison has a common meaning
• Matched population or explained differences Composition changes are visible
• Missing responses reported Results do not silently exclude difficulties
• Critical criteria examined individually Averages do not conceal essential gaps

Consider a hypothetical course with a mean score increase from 40 to 76. The normalized gain is ( 76 − 40 ) / ( 100 − 40 ) = 0.60 (76 - 40)/(100 - 40) = 0.60 ( 76 − 40 ) / ( 100 − 40 ) = 0.60 . That describes the aggregate change under the stated conditions. It does not mean every person gained 36 points or that training alone caused the difference. Examine individual results, instrument quality and other explanations.

Quality and completion-time measures also require comparable tasks and conditions. A faster task performed with different equipment may not indicate learning; a high quality score from an easier specimen set may not be comparable. Define the measure, denominator and expected interpretation before using it in a report.

Combine quantitative and qualitative evidence. Counts identify patterns; learner comments and observations can suggest causes. Distinguish an observed finding from a hypothesis, then test the hypothesis through appropriate course or environment changes. Do not declare success solely from a favorable average when the required performance evidence remains incomplete.

Reporting individual and aggregate evidence

Use aggregate results to identify program patterns, and individual evidence for learner decisions. A class average is not a substitute for checking each person's essential criteria. State how many learners completed both measures and how many still need follow-up.

An evaluation report should also describe the test's scope. Improvement on terminology questions establishes a different outcome from improvement on an observed inspection. Label the evidence accurately and avoid converting one favorable measure into a claim about all learning domains.

Key takeaways

Pre/post comparisons need comparable outcomes, populations and scoring conditions.

Normalized gain is descriptive and does not prove causality or individual mastery.

A pre-test score of 100 makes the normalized-gain denominator zero.