Study guide
Evaluating Countermeasures: Before–After and Cross-Sectional Studies
Road Safety Professional RSP
Track your progress as you read
Finished sections get a check mark.
1 Exam Orientation and Preparation
RSP1 Exam, Eligibility, and Certification Preparing Across the Five RSP1 Domains
2 Foundations: Definitions, Evidence, and Standards
Road Safety Definitions and the Public Health Burden Evaluating Road Safety Literature and Information Nominal and Substantive Safety: Standards and Performance
3 Foundations: Contributing Factors, Energy, and Users
Interacting Crash Contributing Factors Collision Severity, Speed, and Protective Layers Road User Characteristics, Abilities, and Vulnerability
4 Foundations: Partners and Safety Frameworks
Safety Partners, Responsibilities, and Multidisciplinary Perspectives Haddon Matrix and the Development of Safety Management Safe System Principles and Protective Layers Vision Zero, Toward Zero Deaths, and Safety Culture
5 Measurement: Data Collection and Quality
Safety Data Types, Sources, and Users Collecting Quantitative and Qualitative Safety Data The Six Safety Data Quality Attributes Data Process Reviews, Governance, and Investment Decisions
6 Measurement: Exposure, Severity, Models, and Effects
Crash Frequency, Exposure, and Rates Injury Severity, Crash Costs, and Weighted Measures Safety Performance Functions and Suitable Models Regression to the Mean and Empirical Bayes Interpretation CMF Definitions, Scope, and Evidence Quality
7 Human Factors: Perception and the Driving Task
Sensory Perception, Attention, and Workload Driving Task and Perception-Response Time
8 Human Factors: Behavior and Programs
Impairment, Distraction, Fatigue, and Speed Choice Enforcement, Licensing, and Behavioral Countermeasures Outreach, Social Norms, and Audience-Centered Programs
9 Human Factors: Infrastructure, Multimodal Design, and Guidance
Roadway Features, Expectancy, and Behavioral Adaptation Human Factors in Multimodal Planning, Design, and Operations Positive Guidance and Driver Expectancy
10 Problem Solving: Management, Screening, and Diagnosis
The Road Safety Management Process Site-Specific and Systemic Network Screening Crash Patterns, Collision Diagrams, and Diagnostic Hypotheses Field Investigation and Surrogate Safety Measures
11 Problem Solving: Countermeasures and Safety Audits
Countermeasure Sources, Mechanisms, and Selection Combining CMFs and Evaluating Treatment Trade-Offs Road Safety Audits and Documented Owner Responses
12 Problem Solving: Economics and Project Selection
Economic Appraisal: Benefits, Costs, and Discounting Prioritizing Projects and Nonmonetized Impacts
13 Problem Solving: Evaluation Methods and Planning
Evaluating Countermeasures: Before–After and Cross-Sectional Studies Planning Evaluations and Collecting Credible Evidence
14 Implementation: Statewide and Local Plans
Developing Strategic Highway Safety Plans Local, Tribal, and Regional Safety Action Plans
15 Implementation: Funding, Policies, and Practicality
Infrastructure Safety Funding and Program Eligibility Safety Policies, Behavioral Programs, and Practicality
16 Implementation: Teams, Communication, and Learning
Teams, Champions, Leadership, and Organizational Safety Culture Communication, Public Support, and Implementation Consensus Program Monitoring, Safety Targets, and Plan Updates
Evaluating Countermeasures: Before–After and Cross-Sectional Studies
Key Takeaways Explain This Topic AI
A counterfactual estimates what would have occurred without treatment.
Simple before–after comparisons can be biased by regression to the mean and other changes.
EB combines observations and an appropriate SPF, then projects the without-treatment after expectation.
Cross-sectional models require attention to confounding, covariates, applicability, and uncertainty.
Last updated: October 2026
- Evaluating Countermeasures: Before–After and Cross-Sectional Studies
- Estimate the effect against a counterfactual
A treatment evaluation asks what would have happened at the treated sites without the treatment . That unobserved outcome is the counterfactual. Before–after and cross-sectional methods estimate it in different ways. Neither a favorable after-period count nor a plausible mechanism alone establishes a causal treatment effect. The evaluation plan must address exposure, time trends, site selection, reporting, and other changes that could explain the result.
The FHWA Guide to Developing Quality Crash Modification Factors discusses study methods and quality. The CMF Clearinghouse records methods, applicability, and uncertainty for published effects. Use those details when interpreting a claimed percentage reduction. The most credible method depends on the available data and how sites were selected, not on the method's name alone.
Simple before–after: useful description, weak counterfactual
A simple comparison divides observed after crashes by observed before crashes, using comparable periods. If a site has 20 before crashes and 12 after crashes, the raw ratio is 0.60, describing a 40% reduction in observed counts. It does not account for a traffic increase, regional trend, altered reporting threshold, concurrent project, or regression to the mean.
Regression to the mean is especially important when sites were chosen because of unusually high recent counts. Part of the later decrease may have occurred without intervention. Equal before and after durations address one source of imbalance, but do not remove this selection effect. Similarly, dividing both counts by traffic volume does not necessarily solve the problem: safety may have a nonlinear relationship with exposure, and other conditions still change.
Use a simple comparison to report what was observed and identify questions. Avoid calling its ratio an unbiased CMF solely because the computation is straightforward. Short periods and very small counts can also generate unstable percentages; a reduction from two crashes to one is a 50% observed change with substantial uncertainty.
Comparison-group before–after studies
A comparison group estimates background changes affecting untreated sites. Suppose crashes at treated sites decrease while similar untreated sites also decrease. The treatment's net effect is smaller than the treated sites' raw decline if the comparison trend is applicable. Choose comparison sites with relevant similarities in facility type, exposure, environment, and time trends, and ensure they did not receive the treatment or experience spillover effects.
A poorly chosen group can introduce bias. Selecting treated sites for high recent crashes but comparison sites without comparable selection can leave regression-to-the-mean problems. Different land-use changes or reporting systems can invalidate a common-trend assumption. Explain the selection method and inspect trends rather than treating “comparison group included” as proof of study quality.
Empirical Bayes before–after studies
An empirical Bayes (EB) approach combines observed before crashes with an SPF prediction to estimate the expected before-period frequency. It then projects that expectation to the after period under without-treatment conditions, considering the adopted method's exposure and time adjustments. Compare observed after crashes with this projected counterfactual. EB is a strong method for addressing regression to the mean when suitable models and data are available, but it does not automatically eliminate every bias.
Consider an illustrative site with 20 observed before crashes, an SPF prediction of 10, and a supplied overdispersion parameter k = 0.1 k=0.1 k = 0.1 . Using the simplified consistent-period convention w = 1 / ( 1 + k P ) w=1/(1+kP) w = 1/ ( 1 + k P ) gives w = 0.5 w=0.5 w = 0.5 . The EB expected before count is w P + ( 1 − w ) O = 15 wP+(1-w)O=15 wP + ( 1 − w ) O = 15 . Assume the valid without-treatment projection multiplier is 1.2, giving 18 expected after crashes. With 12 observed after crashes, the approximate effect ratio is 12 / 18 = 0.667 12/18=0.667 12/18 = 0.667 , or a 33.3% reduction. The naive observed-count comparison suggested 40%.
This is a teaching calculation, not a complete publication-quality EB estimator. Formal procedures include variance and bias adjustments, pooled sites where appropriate, and uncertainty calculations. The multiplier is supplied here; it is not automatically equal to an AADT growth ratio for every SPF. A poor model, incorrect crash-category match, or unmeasured concurrent treatment can compromise the result despite using EB.
Cross-sectional studies
Cross-sectional studies compare different sites with and without a feature during a study period. A regression model can adjust for measured differences in exposure, geometry, and other covariates. For example, analysts might compare intersections with different control types while adjusting for traffic volumes and configuration.
The challenge is confounding: the feature may have been installed precisely because a site had unusual risks. A coefficient can reflect those selection differences rather than a pure treatment effect. Unmeasured pedestrian activity, land use, or enforcement can also matter. Examine functional form, covariate quality, matching or other adjustment methods, and the range of conditions represented. A cross-sectional association is not automatically transferable to a before–after project decision.
Study designs estimate different comparisons
Simple before–after: compare observed counts across periods.
Comparison-group: use untreated trends to estimate background changes.
EB before–after: combine history and models, then project the counterfactual.
Cross-sectional: compare different sites while addressing confounding.
Interpret uncertainty and report limits
Report the outcome category, sites, periods, treatment implementation, model, effect estimate, and uncertainty. A point estimate below 1.0 can be compatible with no reduction if its interval includes 1.0. An imprecise result does not prove the treatment is ineffective; it may indicate insufficient information. Conversely, statistical significance does not make a small benefit practically important or erase study bias.
Randomization, when feasible and appropriate, can strengthen causal inference; ethical, operational, or funding constraints may limit its use in roadway projects. No design guarantees perfect inference. The sound conclusion explains what the evidence supports, which assumptions are needed, and whether the result applies to the proposed site and crash types.