Study guide

14.3 Statistical Significance, Risk Measures & Evidence-Based Medicine

USMLE Step 2 CK Study Guide Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Human Development & Well-Patient Care

1.1 Infant, Child & Adolescent Development & Growth 1.2 Adult Development, Aging & Geriatric Assessment 1.3 Preventive Health, Immunization & Well-Patient Screening

2 Chapter 2: Immune System & Immunologic Disorders

2.1 Primary & Secondary Immunodeficiencies 2.2 Hypersensitivity Reactions, Allergy & Anaphylaxis 2.3 Autoimmune & Rheumatologic Disease Principles

3 Chapter 3: Blood & Lymphoreticular System

3.1 Anemias: Microcytic, Normocytic & Macrocytic Etiologies 3.2 Bleeding Disorders, Coagulopathies & Platelet Pathology 3.3 Hematologic Malignancies & Lymphoproliferative Disorders

4 Chapter 4: Behavioral Health & Psychiatric Disorders

4.1 Mood Disorders: Major Depression & Bipolar Spectrum 4.2 Anxiety, Trauma-Related & Somatic Symptom Disorders 4.3 Psychotic Disorders & Schizophrenia Spectrum 4.4 Substance Use Disorders & Psychopharmacology

5 Chapter 5: Nervous System & Special Senses

5.1 Cerebrovascular Disease, Stroke & Transients 5.2 Seizures, Epilepsy & Altered Mental Status 5.3 Movement Disorders, Dementia & Peripheral Neuropathies 5.4 Special Senses: Ophthalmologic & Otolaryngologic Emergencies

6 Chapter 6: Musculoskeletal System & Dermatology

6.1 Fractures, Dislocations & Orthopedic Trauma 6.2 Inflammatory Arthropathies & Crystal-Induced Diseases 6.3 Inflammatory & Infectious Dermatoses 6.4 Cutaneous Malignancies & Blistering Disorders

7 Chapter 7: Cardiovascular System

7.1 Coronary Artery Disease & Acute Coronary Syndromes 7.2 Heart Failure, Cardiomyopathies & Valvular Disease 7.3 Cardiac Arrhythmias, Conduction Blocks & EKG Interpretation 7.4 Vascular Emergencies: Aortic Aneurysms & Peripheral Arterial Disease

8 Chapter 8: Respiratory System

8.1 Obstructive Airway Diseases: Asthma & COPD 8.2 Infectious Pneumonias & Pleural Effusions 8.3 Pulmonary Vascular Disease & Thromboembolism 8.4 Acute Respiratory Distress Syndrome & Interstitial Lung Diseases

9 Chapter 9: Gastrointestinal System

9.1 Esophageal, Gastric & Peptic Ulcer Disorders 9.2 Inflammatory Bowel Disease & Colorectal Neoplasms 9.3 Hepatic Disorders, Cirrhosis & Portal Hypertension 9.4 Biliary Tract Diseases & Pancreatitis

10 Chapter 10: Renal, Urinary & Reproductive Systems

10.1 Acute Kidney Injury & Chronic Kidney Disease 10.2 Glomerular & Tubulointerstitial Diseases 10.3 Nephrolithiasis, Urinary Tract Infections & Prostate Pathology 10.4 Fluid, Electrolyte & Acid-Base Disorders 10.5 Female Reproductive Disorders & Gynecology 10.6 Breast Pathology & Clinical Breast Disease

11 Chapter 11: Pregnancy, Childbirth & Puerperium

11.1 Normal Prenatal Care & Maternal-Fetal Physiology 11.2 Medical & Obstetric Complications of Pregnancy 11.3 Labor, Delivery & Postpartum Management

12 Chapter 12: Endocrine System

12.1 Diabetes Mellitus, Hypoglycemia & Metabolic Emergencies 12.2 Thyroid & Parathyroid Disorders 12.3 Adrenal & Pituitary Axis Pathology

13 Chapter 13: Multisystem Processes & Critical Care

13.1 Shock States, Sepsis & Systemic Inflammatory Response 13.2 Environmental Emergencies, Toxicology & Overdose 13.3 Infectious Disease Principles & Systemic Pathogens

14 Chapter 14: Biostatistics, Epidemiology & Literature Interpretation

14.1 Study Designs, Bias & Validity 14.2 Diagnostic Testing: Sensitivity, Specificity, PPV & NPV 14.3 Statistical Significance, Risk Measures & Evidence-Based Medicine

15 Chapter 15: Social Sciences, Ethics & Systems-Based Practice

15.1 Medical Ethics, Informed Consent & Patient Autonomy 15.2 Legal Issues, Confidentiality & Surrogate Decision Making 15.3 Patient Safety, Quality Improvement & Medical Error Analysis 15.4 Health Care Delivery Systems, Interprofessional Care & Transitions

14.3 Statistical Significance, Risk Measures & Evidence-Based Medicine

Key Takeaways Explain This Topic AI

Absolute Risk Reduction (ARR = Risk_control - Risk_treatment) determines the Number Needed to Treat (NNT = 1 / ARR), rounding up to the nearest whole integer.

Relative Risk (RR = Risk_exposed / Risk_unexposed) measures strength of association in cohort studies; Relative Risk Reduction (RRR = 1 - RR = ARR / Risk_control).

Type I error (alpha, typically set at 0.05) is rejecting a true null hypothesis (false positive); Type II error (beta) is failing to reject a false null hypothesis (false negative).

Statistical Power (1 - beta, target >= 0.80) increases with larger sample size, larger effect size, lower measurement variance, and higher significance threshold (alpha).

Confidence intervals (95% CI) that include 1.0 for ratios (RR, OR, HR) or 0.0 for mean differences indicate lack of statistical significance at the alpha = 0.05 level.

Last updated: July 2026

  • 14.3 Statistical Significance, Risk Measures & Evidence-Based Medicine
  • Measures of Risk & Clinical Impact

In clinical trial appraisal, quantifying treatment effect requires understanding both relative and absolute risk metrics. Relative measures express proportional change, whereas absolute measures incorporate baseline disease frequency, providing the true clinical impact of an intervention.

Event Rates

Experimental Event Rate (EER) or I e I_e I e ​ : Incidence of the outcome in the intervention group = a a + b = \frac{a}{a + b} = a + b a ​ .

Control Event Rate (CER) or I c I_c I c ​ : Incidence of the outcome in the control group = c c + d = \frac{c}{c + d} = c + d c ​ .

Relative Risk (RR) and Relative Risk Reduction (RRR)

Relative Risk (RR) : Ratio of disease incidence in the treated/exposed group to incidence in the control group: R R = E E R C E R = a / ( a + b ) c / ( c + d ) RR = \frac{EER}{CER} = \frac{a / (a + b)}{c / (c + d)} RR = CER EER ​ = c / ( c + d ) a / ( a + b ) ​

R R = 1.0 RR = 1.0 RR = 1.0 : No difference in risk between groups.

R R < 1.0 RR < 1.0 RR < 1.0 : Intervention reduces risk (protective effect).

R R > 1.0 RR > 1.0 RR > 1.0 : Intervention increases risk (harmful effect).

Relative Risk Reduction (RRR) : Proportional reduction in risk achieved by the intervention: R R R = C E R − E E R C E R = 1 − R R RRR = \frac{CER - EER}{CER} = 1 - RR RRR = CER CER − EER ​ = 1 − RR

Absolute Risk Reduction (ARR) & Number Needed to Treat (NNT)

Absolute Risk Reduction (ARR) : The true difference in absolute disease risk between control and intervention groups: A R R = ∣ C E R − E E R ∣ ARR = |CER - EER| A RR = ∣ CER − EER ∣

Number Needed to Treat (NNT) : The number of patients who must receive the treatment for one additional patient to experience a beneficial outcome (or prevent one adverse event): N N T = 1 A R R = 1 C E R − E E R NNT = \frac{1}{ARR} = \frac{1}{CER - EER} NNT = A RR 1 ​ = CER − EER 1 ​

Rounding Rule : Always round NNT UP to the next whole integer to avoid overestimating clinical benefit (e.g., N N T = 14.2 → 15 NNT = 14.2 \rightarrow 15 NNT = 14.2 → 15 ).

Number Needed to Harm (NNH)

When an exposure or therapy increases adverse events, the Absolute Risk Increase (ARI) is E E R − C E R EER - CER EER − CER . The Number Needed to Harm (NNH) represents the number of patients exposed to the risk factor for one additional patient to experience harm: N N H = 1 A R I = 1 E E R − C E R NNH = \frac{1}{ARI} = \frac{1}{EER - CER} NN H = A R I 1 ​ = EER − CER 1 ​

Rounding Rule : Always round NNH DOWN to the next whole integer to avoid underestimating risk of harm.

Metric Formula Clinical Interpretation

Relative Risk (RR) E E R / C E R EER / CER EER / CER Proportional risk ratio; independent of baseline incidence

Relative Risk Reduction (RRR) ( C E R − E E R ) / C E R (CER - EER) / CER ( CER − EER ) / CER Percentage reduction in risk relative to baseline control rate

Absolute Risk Reduction (ARR) C E R − E E R CER - EER CER − EER True absolute difference in disease incidence between arms

Number Needed to Treat (NNT) 1 / A R R 1 / ARR 1/ A RR Patients treated to prevent 1 event; round UP

Number Needed to Harm (NNH) 1 / ( E E R − C E R ) 1 / (EER - CER) 1/ ( EER − CER ) Patients exposed for 1 adverse event to occur; round DOWN

Hypothesis Testing: Errors & Power

Hypothesis testing evaluates whether observed sample differences reflect true population effects or mere random chance.

Null Hypothesis ( H 0 H_0 H 0 ​ ) : Statement of no difference or no association between groups.

Alternative Hypothesis ( H 1 H_1 H 1 ​ ) : Statement that a true difference or association exists.

Study Conclusion Real-World Truth: H 0 H_0 H 0 ​ True (No Effect) Real-World Truth: H 1 H_1 H 1 ​ True (Real Effect)

Reject H 0 H_0 H 0 ​ (Find Effect) Type I Error ( α \alpha α ) (False Positive) Correct Decision ( 1 − β 1 - \beta 1 − β ) ( Power )

Fail to Reject H 0 H_0 H 0 ​ (No Effect) Correct Decision ( 1 − α 1 - \alpha 1 − α ) Type II Error ( β \beta β ) (False Negative)

Type I Error ( α \alpha α )

A Type I error occurs when researchers reject the null hypothesis when H 0 H_0 H 0 ​ is actually true (finding a false positive difference). The significance level α \alpha α (typically set at 0.05 0.05 0.05 or 5 % 5\% 5% ) is the maximum acceptable probability of committing a Type I error. The calculated p p p -value represents the probability of obtaining results at least as extreme as observed, assuming H 0 H_0 H 0 ​ is true. If p ≤ α p \le \alpha p ≤ α , H 0 H_0 H 0 ​ is rejected.

Type II Error ( β \beta β ) & Statistical Power

A Type II error occurs when researchers fail to reject H 0 H_0 H 0 ​ when a true treatment difference exists (a false negative study).

Statistical Power ( 1 − β 1 - \beta 1 − β ) : The probability that a study will correctly detect a true effect of a specified size. Target power in clinical trials is usually ≥ 0.80 \ge 0.80 ≥ 0.80 ( 80 % 80\% 80% ).

Determinants of Power :

Sample Size ( n n n ) : Larger sample size reduces standard error, directly increasing power .

Effect Size : Larger true differences between groups are easier to detect, increasing power .

Measurement Variance ( σ 2 \sigma^2 σ 2 ) : Lower data scatter/SD increases power .

Significance Level ( α \alpha α ) : Increasing α \alpha α (e.g., 0.01 → 0.05 0.01 \rightarrow 0.05 0.01 → 0.05 ) increases power (though increases Type I error risk).

Confidence Intervals & Statistical Significance

A 95% Confidence Interval (CI) provides a range of plausible values for a population parameter, constructed such that 95 % 95\% 95% of such intervals contain the true parameter value. 95% CI = Point Estimate ± ( 1.96 × Standard Error ) \text{95\% CI} = \text{Point Estimate} \pm (1.96 \times \text{Standard Error}) 95% CI = Point Estimate ± ( 1.96 × Standard Error ) Standard Error (SE) = Standard Deviation (SD) n \text{Standard Error (SE)} = \frac{\text{Standard Deviation (SD)}}{\sqrt{n}} Standard Error (SE) = n ​ Standard Deviation (SD) ​

Determining Statistical Significance from CIs

For Ratio Metrics (RR, Odds Ratio, Hazard Ratio) :

The null value of no effect is 1.0 .

If the 95% CI includes 1.0 (e.g., R R = 0.82 RR = 0.82 RR = 0.82 , 95 % CI: 0.65 − 1.08 95\%\text{ CI: } 0.65 - 1.08 95% CI: 0.65 − 1.08 ), the result is not statistically significant ( p > 0.05 p > 0.05 p > 0.05 ).

If the 95% CI excludes 1.0 (e.g., R R = 0.74 RR = 0.74 RR = 0.74 , 95 % CI: 0.58 − 0.94 95\%\text{ CI: } 0.58 - 0.94 95% CI: 0.58 − 0.94 ), the result is statistically significant ( p < 0.05 p < 0.05 p < 0.05 ).

For Mean Differences / Absolute Risk Differences (ARR) :

The null value of no effect is 0.0 .

If the 95% CI includes 0.0 (e.g., Mean Diff = − 1.4 = -1.4 = − 1.4 , 95 % CI: − 3.2 to + 0.4 95\%\text{ CI: } -3.2 \text{ to } +0.4 95% CI: − 3.2 to + 0.4 ), the result is not statistically significant ( p > 0.05 p > 0.05 p > 0.05 ).

If the 95% CI excludes 0.0 (e.g., Mean Diff = − 3.8 = -3.8 = − 3.8 , 95 % CI: − 5.6 to − 2.0 95\%\text{ CI: } -5.6 \text{ to } -2.0 95% CI: − 5.6 to − 2.0 ), the result is statistically significant ( p < 0.05 p < 0.05 p < 0.05 ).

Selecting the Correct Statistical Test

Selecting appropriate statistical tests requires identifying the variable types (continuous vs. categorical) and group arrangements.

Statistical Test Selection Guide

Comparing 2 Independent Group Means :

Two-Sample t t t -test (Unpaired t t t -test) : Compares means of a continuous dependent variable across 2 categorical groups (e.g., mean blood pressure in treatment vs. placebo).

Comparing ≥ 3 \ge 3 ≥ 3 Independent Group Means :

ANOVA (Analysis of Variance) : Compares continuous means across 3 or more categorical groups (e.g., mean HbA1c across placebo, low-dose, and high-dose drug arms).

Comparing Paired Continuous Measurements :

Paired t t t -test : Compares continuous measurements before and after an intervention in the same individuals (e.g., pre- vs post-treatment serum cholesterol).

Comparing Categorical Proportions :

Chi-Square ( χ 2 \chi^2 χ 2 ) Test : Evaluates associations between 2 categorical variables (e.g., proportion of smokers vs non-smokers who develop stroke).

Fisher Exact Test : Used instead of Chi-Square when expected cell counts in a 2 × 2 2 \times 2 2 × 2 table are small ( < 5 < 5 < 5 ).

Correlation & Regression :

Pearson Correlation Coefficient ( r r r ) : Measures linear association between 2 continuous variables (ranges from − 1.0 -1.0 − 1.0 to + 1.0 +1.0 + 1.0 ).

Linear Regression : Predicts a continuous outcome from continuous/categorical predictors.

Logistic Regression : Predicts a binary categorical outcome (e.g., dead/alive) from predictors.

  • Independent Variable Dependent Variable Number of Groups Recommended Statistical Test
  • Categorical Continuous 2 Independent groups Two-Sample (Unpaired) t t t -test
  • Categorical Continuous ≥ 3 \ge 3 ≥ 3 Independent groups ANOVA (Analysis of Variance)
  • Categorical Continuous Paired (Pre/Post in same subject) Paired t t t -test
  • Categorical Categorical 2 or more groups (cell counts ≥ 5 \ge 5 ≥ 5 ) Chi-Square ( χ 2 \chi^2 χ 2 ) test
  • Categorical Categorical 2 groups (cell counts < 5 < 5 < 5 ) Fisher Exact test
  • Continuous Continuous N/A Pearson Correlation ( r r r ) / Linear Regression
  • Evidence-Based Medicine Decision Algorithm

Calculate Clinical Risk Reduction :

Given E E R EER EER and C E R CER CER → \rightarrow → Calculate A R R = C E R − E E R ARR = CER - EER A RR = CER − EER , then N N T = 1 A R R NNT = \frac{1}{ARR} NNT = A RR 1 ​ (round UP).

Evaluate 95% Confidence Interval :

Ratio metric → \rightarrow → Check if interval crosses 1.0 . If crossed → p > 0.05 \rightarrow p > 0.05 → p > 0.05 .

Difference metric → \rightarrow → Check if interval crosses 0.0 . If crossed → p > 0.05 \rightarrow p > 0.05 → p > 0.05 .

Choose Statistical Test :

Continuous outcome across 2 groups → \rightarrow → t t t -test .

Continuous outcome across ≥ 3 \ge 3 ≥ 3 groups → \rightarrow → ANOVA .

Categorical outcome across groups → \rightarrow → Chi-Square .