Study guide
6.2 Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values
PEBC Pharmacist Qualifying Examination Part I MCQ
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Canadian Pharmacy Regulatory Framework & Federal Legislation
1.1 Food and Drugs Act, Regulations, and Health Canada Oversight 1.2 Controlled Drugs and Substances Act (CDSA) & Controlled Substances Regulations (CSR) 1.3 NAPRA Drug Scheduling Framework & National Model Standards of Practice
2 Chapter 2: Ethics, Scope of Practice & Safety Culture
2.1 Professional Ethics, Fiduciary Duty & Health Information Privacy 2.2 Expanded Scope of Practice: Minor Ailments Prescribing, Renewals & Injections 2.3 Equity, Diversity, Inclusion, Cultural Safety & Continuous Quality Improvement
3 Chapter 3: Pharmaceutical Calculations & Posology
3.1 Concentrations, Dilutions, Alligations & Reconstitution 3.2 Dosing by Weight/BSA, Days Supply & Pediatric/Geriatric Dose Adjustments 3.3 IV Infusion Rates, Drop Factors, Osmolarity & Creatinine Clearance (Cockcroft-Gault)
4 Chapter 4: Compounding, Dispensing & Product Integrity
4.1 Non-Sterile Compounding & Beyond-Use Dating (NAPRA Standards) 4.2 Sterile Preparations & Hazardous Drug Handling (NAPRA / NIOSH Principles) 4.3 Prescription Verification, Auxiliary Labeling, Cold Chain & Medication Incident Prevention (ISMP Canada) 4.4 Dispensing Operations: Interchangeability, Adjudication, Compliance Packaging, Release & Inventory Control
5 Chapter 5: Clinical Pharmacology & Pharmacokinetics
5.1 Pharmacokinetic Principles: Clearance, Volume of Distribution, Half-life & Steady State 5.2 Cytochrome P450 Enzymes, Transporters & Significant Drug-Drug Interactions 5.3 Therapeutic Drug Monitoring (TDM), Narrow Therapeutic Index Drugs & Toxicities
6 Chapter 6: Evidence-Based Medicine & Literature Appraisal
6.1 Study Designs, Bias & Levels of Evidence in Clinical Research 6.2 Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values 6.3 Applying Canadian Practice Guidelines to Clinical Decision-Making 6.4 Drug Information Practice, Diagnostic Test Accuracy & Pharmacoeconomics
7 Chapter 7: Cardiovascular Pharmacotherapy: Hypertension, Dyslipidemia & CAD
7.1 Hypertension Management (Hypertension Canada Guidelines, Targets & Regimens) 7.2 Dyslipidemia & Atherosclerotic Cardiovascular Disease (CCS Guidelines & Statin Therapy) 7.3 Ischemic Heart Disease & Acute Coronary Syndromes (Antiplatelet & Antianginal Therapy)
8 Chapter 8: Cardiovascular Pharmacotherapy: Heart Failure, Arrhythmias & Anticoagulation
8.1 Heart Failure with Reduced & Preserved Ejection Fraction (GDMT & Monitoring) 8.2 Atrial Fibrillation, Arrhythmias & Rate vs. Rhythm Control 8.3 Venous Thromboembolism & Anticoagulation Therapy (DOACs, Warfarin & Thrombosis Canada)
9 Chapter 9: Endocrine & Metabolic Disorders
9.1 Type 2 Diabetes: Glycemic Targets, Non-Insulin Pharmacotherapy (Diabetes Canada) 9.2 Insulin Regimens, Type 1 Diabetes, Hypoglycemia & Diabetic Emergencies (DKA/HHS) 9.3 Thyroid Disorders & Osteoporosis (Diagnostic Parameters & Therapeutic Management)
10 Chapter 10: Respiratory & Gastrointestinal Disorders
10.1 Asthma Management & Exacerbations (CTS Guidelines, Relievers & Controllers) 10.2 Chronic Obstructive Pulmonary Disease (COPD) & Inhaler Delivery Systems 10.3 Gastroesophageal Reflux Disease (GERD), Peptic Ulcer Disease & GI Bleeding 10.4 Lower GI & Liver Disorders: Constipation, Diarrhea, IBS, IBD & Cirrhosis
11 Chapter 11: Infectious Diseases & Antimicrobial Stewardship
11.1 Respiratory Tract Infections: Otitis, Sinusitis, Pharyngitis, CAP & HAP 11.2 Urinary Tract, Skin/Soft Tissue & Sexually Transmitted Infections 11.3 Systemic Infections, Clostridioides difficile & Antimicrobial Stewardship Principles 11.4 HIV Prevention & Treatment, Viral Hepatitis, Tuberculosis & Fungal Infections
12 Chapter 12: Central Nervous System & Mental Health
12.1 Major Depressive Disorder & Anxiety Disorders (First-Line Agents & Safety) 12.2 Bipolar Disorder, Schizophrenia & Psychopharmacology Monitoring 12.3 Epilepsy, Seizure Types, Antiseizure Medications & Neuropathic Pain 12.4 Dementia, Parkinson Disease, Migraine, Insomnia & Restless Legs Syndrome
13 Chapter 13: Special Populations, Pain Management & Substance Use
13.1 Acute & Chronic Pain Management: Multimodal Analgesia & Opioid Stewardship 13.2 Substance Use Disorders: Opioid Agonist Therapy, Harm Reduction & Alcohol Withdrawal 13.3 Special Populations: Pharmacotherapy in Pregnancy, Lactation & Geriatrics (Deprescribing / Beers) 13.4 Palliative Care & Pediatric Pharmacotherapy
14 Chapter 14: Renal, Rheumatologic, Hematologic & Oncology Supportive Care
14.1 Chronic Kidney Disease, Acute Kidney Injury & Electrolyte Disorders 14.2 Osteoarthritis, Rheumatoid Arthritis & Gout 14.3 Anemias & Oncology Supportive Care
15 Chapter 15: Minor Ailments, Dermatology and Women's & Men's Health
15.1 Minor Ailments I: Allergic Rhinitis, Colds & Influenza, Eye and Ear Conditions 15.2 Minor Ailments II: Acne, Dermatitis, Psoriasis, Lice, Scabies & Sun Protection 15.3 Women's & Men's Health: Contraception, Emergency Contraception, Menopause, BPH, ED & Overactive Bladder
16 Chapter 16: Immunization & Health Promotion
16.1 Immunization Practice: NACI Recommendations, Administration, Cold Chain & Adverse Events 16.2 Health Promotion: Smoking Cessation, Weight Management, Screening & Social Determinants of Health
17 Chapter 17: Patient Care Process, Communication & Stewardship
17.1 The Patient Care Process: Assessment, Care Planning & Patient Interviewing 17.2 Patient Education, Device Counselling, Health Literacy & Overcoming Barriers 17.3 Interprofessional Collaboration, Pharmacy Workflow, Team Delegation & Drug Shortage Stewardship
6.2 Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values
Key Takeaways Explain This Topic AI
Absolute Risk Reduction ( A R R = ∣ C E R − E E R ∣ ARR = |CER - EER| A RR = ∣ CER − EER ∣ ) quantifies the true clinical magnitude of treatment benefit in the population, whereas Relative Risk Reduction ( R R R = A R R C E R = 1 − R R RRR = \frac{ARR}{CER} = 1 - RR RRR = CER A RR = 1 − RR ) inflates clinical perception by expressing proportional reduction regardless of baseline risk.
Number Needed to Treat ( N N T = 1 A R R NNT = \frac{1}{ARR} NNT = A RR 1 ) is calculated by dividing 1 by the ARR in decimal form and must always be rounded UP to the next whole integer to prevent overestimating clinical benefit over a specified treatment timeframe.
Number Needed to Harm ( N N H = 1 A R I NNH = \frac{1}{ARI} NN H = A R I 1 ) quantifies adverse event incidence from an Absolute Risk Increase ( A R I = E E R − C E R ARI = EER - CER A R I = EER − CER ) and must always be rounded DOWN to the previous whole integer to avoid underestimating toxic harm.
In hypothesis testing, Type I error ( α \alpha α , typically set at 0.05 0.05 0.05 ) represents the probability of falsely rejecting a true null hypothesis (false positive), whereas Type II error ( β \beta β , typically 0.10 0.10 0.10 to 0.20 0.20 0.20 ) represents failing to reject a false null hypothesis (false negative), giving statistical power ( 1 − β 1 - \beta 1 − β , ≥ 80 % \ge 80\% ≥ 80% ).
A 95% Confidence Interval (CI) indicates statistical significance at p < 0.05 p < 0.05 p < 0.05 if it excludes the value of no effect: for difference metrics ( A R R ARR A RR , mean difference), the CI must not cross 0; for ratio metrics ( R R RR RR , O R OR OR , H R HR H R ), the CI must not cross 1.0.
Last updated: October 2026
Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values
Biostatistics provides the quantitative tools required to interpret therapeutic trial results, separate true pharmacological effects from random variation, and translate research data into actionable patient care decisions. For Canadian pharmacists, evaluating drug promotional materials, manufacturer claims, and clinical guidelines requires fluent mastery of risk metrics, hypothesis testing, error boundaries, and confidence interval interpretation.
Quantitative Clinical Trial Metrics: The 2x2 Table
Most binary clinical trial outcomes (such as stroke vs. no stroke, mortality vs. survival, or adverse reaction vs. no reaction) can be organized into a standard 2 × 2 2 \times 2 2 × 2 contingency table:
Treatment Group Primary Event Occurred Primary Event Did Not Occur Total Participants Event Rate Formula
Experimental Arm a a a b b b a + b a + b a + b Experimental Event Rate ( E E R ) = a a + b \text{Experimental Event Rate } (EER) = \frac{a}{a+b} Experimental Event Rate ( EER ) = a + b a
Control / Placebo Arm c c c d d d c + d c + d c + d Control Event Rate ( C E R ) = c c + d \text{Control Event Rate } (CER) = \frac{c}{c+d} Control Event Rate ( CER ) = c + d c
From these foundational event rates, clinical biostatisticians derive relative and absolute effect metrics.
- Relative Risk, Absolute Risk Reduction, and Relative Risk Reduction
- 1. Relative Risk (Risk Ratio, R R RR RR )
Relative Risk is the ratio of the probability of an outcome occurring in the experimental group compared to the control group:
R R = E E R C E R RR = \frac{EER}{CER} RR = CER EER
Interpretation:
R R = 1.0 RR = 1.0 RR = 1.0 : There is no difference in risk between the experimental and control arms (the null value).
R R < 1.0 RR < 1.0 RR < 1.0 : The experimental intervention decreases risk (protective effect).
R R > 1.0 RR > 1.0 RR > 1.0 : The experimental intervention increases risk (harmful effect).
2. Absolute Risk Reduction ( A R R ARR A RR )
Absolute Risk Reduction (also termed Risk Difference) represents the absolute arithmetic difference in event rates between the two groups:
A R R = ∣ C E R − E E R ∣ = C E R − E E R ( when C E R > E E R ) ARR = |CER - EER| = CER - EER \quad (\text{when } CER > EER) A RR = ∣ CER − EER ∣ = CER − EER ( when CER > EER )
A R R ARR A RR directly reflects the true clinical impact in the population, answering the question: "Out of 100 patients treated with this therapy, how many will be spared an adverse clinical event?"
3. Relative Risk Reduction ( R R R RRR RRR )
Relative Risk Reduction represents the proportional reduction in risk achieved by the experimental intervention relative to baseline control risk:
R R R = 1 − R R = C E R − E E R C E R = A R R C E R RRR = 1 - RR = \frac{CER - EER}{CER} = \frac{ARR}{CER} RRR = 1 − RR = CER CER − EER = CER A RR
Important
The Baseline Risk Fallacy (Why RRR Distorts Clinical Reality): Pharmaceutical marketing frequently highlights R R R RRR RRR rather than A R R ARR A RR because R R R RRR RRR yields impressively large percentages that remain constant regardless of whether the baseline risk is massive or negligible.
Consider two clinical trials, both reporting an identical R R R = 25 % RRR = 25\% RRR = 25% :
High-Risk Population: C E R = 20.0 % CER = 20.0\% CER = 20.0% , E E R = 15.0 % EER = 15.0\% EER = 15.0% .
- A R R = 20.0 % − 15.0 % = 5.0 % ARR = 20.0\% - 15.0\% = 5.0\% A RR = 20.0% − 15.0% = 5.0%
- N N T = 1 0.05 = 20 NNT = \frac{1}{0.05} = 20 NNT = 0.05 1 = 20
Low-Risk Population: C E R = 0.20 % CER = 0.20\% CER = 0.20% , E E R = 0.15 % EER = 0.15\% EER = 0.15% .
- A R R = 0.20 % − 0.15 % = 0.05 % ARR = 0.20\% - 0.15\% = 0.05\% A RR = 0.20% − 0.15% = 0.05%
- N N T = 1 0.0005 = 2 , 000 NNT = \frac{1}{0.0005} = 2,000 NNT = 0.0005 1 = 2 , 000
While both therapies achieve the exact same " 25 % 25\% 25% relative risk reduction," treating high-risk patients requires treating only 20 individuals to prevent an event, whereas treating low-risk patients requires treating 2,000 individuals. Pharmacists must always calculate A R R ARR A RR and N N T NNT NNT to discern true clinical utility.
- Number Needed to Treat (NNT) and Number Needed to Harm (NNH)
- Number Needed to Treat ( N N T NNT NNT )
N N T NNT NNT is the number of patients who must receive a specific therapeutic intervention for a defined period of time for one additional patient to experience a beneficial clinical outcome (or avoid an adverse event) compared to control:
- N N T = 1 A R R = 1 C E R − E E R NNT = \frac{1}{ARR} = \frac{1}{CER - EER} NNT = A RR 1 = CER − EER 1
- (Note: A R R ARR A RR must be entered in decimal form, e.g., 5 % = 0.05 5\% = 0.05 5% = 0.05 )
Mandatory Rounding Convention for N N T NNT NNT :
Always round UP to the next whole integer.
Mathematical Rationale: If N N T NNT NNT calculates to 23.1 23.1 23.1 or 23.8 23.8 23.8 , treating 23 patients will prevent fewer than 1 full event ( 23 × 0.042 = 0.966 23 \times 0.042 = 0.966 23 × 0.042 = 0.966 events prevented). To ensure at least one full event is prevented, 24 patients must be treated. Rounding down would overestimate therapeutic benefit.
Temporal Requirement: N N T NNT NNT is clinically meaningless without an explicit timeframe. An N N T NNT NNT of 25 over 1 year represents far greater clinical benefit than an N N T NNT NNT of 25 over 10 years.
Absolute Risk Increase ( A R I ARI A R I ) and Number Needed to Harm ( N N H NNH NN H )
When an intervention increases the incidence of an adverse drug event or toxicity, the difference in event rates is the Absolute Risk Increase ( A R I ARI A R I ) :
A R I = E E R − C E R ( when E E R > C E R ) ARI = EER - CER \quad (\text{when } EER > CER) A R I = EER − CER ( when EER > CER )
Number Needed to Harm ( N N H NNH NN H ) indicates the number of patients who must be exposed to the intervention for a defined period before one additional patient experiences the adverse event:
N N H = 1 A R I = 1 E E R − C E R NNH = \frac{1}{ARI} = \frac{1}{EER - CER} NN H = A R I 1 = EER − CER 1
Mandatory Rounding Convention for N N H NNH NN H :
Always round DOWN to the previous whole integer.
Clinical Safety Rationale: If N N H NNH NN H calculates to 52.8 52.8 52.8 , treating 52 patients already exposes the cohort to the toxic threshold. To maintain conservative vigilance and avoid underestimating harm, round down to 52. Rounding up to 53 would falsely suggest that patients can tolerate higher exposure before experiencing toxicity.
Step-by-Step Worked Clinical Biostatistics Problem
Clinical Trial Scenario: A 3-year randomized, double-blind trial investigates a novel SGLT2 inhibitor versus placebo in 5,000 patients with heart failure with reduced ejection fraction ( 2 , 500 2,500 2 , 500 patients per arm).
Primary Efficacy Outcome (CV Death or HF Hospitalization):
Placebo arm: 345 345 345 events out of 2 , 500 2,500 2 , 500 patients.
Treatment arm: 240 240 240 events out of 2 , 500 2,500 2 , 500 patients.
Primary Safety Outcome (Symptomatic Mycotic Genital Infections):
Treatment arm: 68 68 68 events out of 2 , 500 2,500 2 , 500 patients.
Placebo arm: 20 20 20 events out of 2 , 500 2,500 2 , 500 patients.
Step-by-Step Efficacy Calculations:
Calculate Control and Experimental Event Rates: C E R = 345 2 , 500 = 0.138 ( 13.8 % i m p l i c a t i o n ) CER = \frac{345}{2,500} = 0.138 \quad (13.8\% implication) CER = 2 , 500 345 = 0.138 ( 13.8% im pl i c a t i o n ) E E R = 240 2 , 500 = 0.096 ( 9.6 % i m p l i c a t i o n ) EER = \frac{240}{2,500} = 0.096 \quad (9.6\% implication) EER = 2 , 500 240 = 0.096 ( 9.6% im pl i c a t i o n )
Calculate Relative Risk ( R R RR RR ): R R = E E R C E R = 0.096 0.138 = 0.6957 ≈ 0.70 RR = \frac{EER}{CER} = \frac{0.096}{0.138} = 0.6957 \approx 0.70 RR = CER EER = 0.138 0.096 = 0.6957 ≈ 0.70
Calculate Absolute Risk Reduction ( A R R ARR A RR ): A R R = C E R − E E R = 0.138 − 0.096 = 0.042 ( 4.2 % ) ARR = CER - EER = 0.138 - 0.096 = 0.042 \quad (4.2\%) A RR = CER − EER = 0.138 − 0.096 = 0.042 ( 4.2% )
Calculate Relative Risk Reduction ( R R R RRR RRR ): R R R = A R R C E R = 0.042 0.138 = 0.3043 ( 30.43 % ≈ 30.4 % ) RRR = \frac{ARR}{CER} = \frac{0.042}{0.138} = 0.3043 \quad (30.43\% \approx 30.4\%) RRR = CER A RR = 0.138 0.042 = 0.3043 ( 30.43% ≈ 30.4% )
Calculate N N T NNT NNT : N N T = 1 A R R = 1 0.042 = 23.81 NNT = \frac{1}{ARR} = \frac{1}{0.042} = 23.81 NNT = A RR 1 = 0.042 1 = 23.81
Applying the round-up rule: N N T = 24 NNT = 24 NNT = 24 over 3 years.
Step-by-Step Safety Calculations:
Calculate Safety Event Rates: E E R harm = 68 2 , 500 = 0.0272 ( 2.72 % ) EER_{\text{harm}} = \frac{68}{2,500} = 0.0272 \quad (2.72\%) EE R harm = 2 , 500 68 = 0.0272 ( 2.72% ) C E R harm = 20 2 , 500 = 0.0080 ( 0.80 % ) CER_{\text{harm}} = \frac{20}{2,500} = 0.0080 \quad (0.80\%) CE R harm = 2 , 500 20 = 0.0080 ( 0.80% )
Calculate Absolute Risk Increase ( A R I ARI A R I ): A R I = E E R harm − C E R harm = 0.0272 − 0.0080 = 0.0192 ( 1.92 % ) ARI = EER_{\text{harm}} - CER_{\text{harm}} = 0.0272 - 0.0080 = 0.0192 \quad (1.92\%) A R I = EE R harm − CE R harm = 0.0272 − 0.0080 = 0.0192 ( 1.92% )
Calculate N N H NNH NN H : N N H = 1 A R I = 1 0.0192 = 52.08 NNH = \frac{1}{ARI} = \frac{1}{0.0192} = 52.08 NN H = A R I 1 = 0.0192 1 = 52.08
Applying the round-down rule: N N H = 52 NNH = 52 NN H = 52 over 3 years.
Clinical Synthesis: Over 3 years, for every 24 patients treated with the novel drug instead of placebo, 1 cardiovascular death or heart failure hospitalization is prevented ( N N T = 24 NNT = 24 NNT = 24 ), while 1 additional mycotic infection occurs for every 52 patients treated ( N N H = 52 NNH = 52 NN H = 52 ).
- Odds Ratio (OR) vs. Hazard Ratio (HR)
- Odds Ratio ( O R OR OR )
In retrospective case-control studies where total population denominator counts are unavailable, incidence cannot be measured, precluding calculation of Relative Risk. Instead, researchers compute the Odds Ratio ( O R OR OR ) :
O R = Odds of exposure among cases Odds of exposure among controls = a / c b / d = a × d b × c OR = \frac{\text{Odds of exposure among cases}}{\text{Odds of exposure among controls}} = \frac{a / c}{b / d} = \frac{a \times d}{b \times c} OR = Odds of exposure among controls Odds of exposure among cases = b / d a / c = b × c a × d
Note
The Rare Disease Assumption: When the clinical outcome of interest is rare in the general population (incidence < 5 % < 5\% < 5% to 10 % 10\% 10% ), the Odds Ratio closely approximates the Relative Risk ( O R ≈ R R OR \approx RR OR ≈ RR ). However, when the outcome is common, the Odds Ratio substantially diverges and overstates the Relative Risk (exaggerating both protective and harmful effects).
Hazard Ratio ( H R HR H R )
A Hazard Ratio is derived from survival (time-to-event) analysis, typically using Cox proportional hazards regression modeling. Unlike Relative Risk—which simply counts whether an event occurred by the end of a trial—the Hazard Ratio compares the instantaneous event rate occurring at any given point in time during follow-up, accounting for:
- Varying durations of patient follow-up ;
- Censoring (participants who complete the study without an event, withdraw, or are lost to follow-up);
The timing of events (detecting whether a drug delays events early even if lifetime cumulative incidence converges).
Interpretation:
H R = 1.0 HR = 1.0 H R = 1.0 : Equivalent instantaneous event hazard in both arms (null value).
H R < 1.0 HR < 1.0 H R < 1.0 : Experimental therapy reduces the hazard rate over time (protective).
H R > 1.0 HR > 1.0 H R > 1.0 : Experimental therapy accelerates the hazard rate over time (hazardous).
Statistical Hypothesis Testing: Type I Error, Type II Error, and Power
Clinical trials utilize inferential statistics to test formal hypotheses regarding whether observed differences reflect true pharmacological reality or mere random sampling fluctuation. THE HYPOTHESIS TESTING MATRIX
- TRUE STATE OF REALITY
- ┌───────────────────────┬───────────────────────┐
- │ H₀ is TRUE │ H₀ is FALSE │
- │ (No True Difference) │ (True Difference Exists)│
- ┌───────────────────────────────┼───────────────────────┼───────────────────────┤
- │ REJECT H₀ │ TYPE I ERROR (α) │ CORRECT DECISION │
- │ (Conclude Difference Exists) │ "False Positive" │ POWER (1 - β) │
- ├───────────────────────────────┼───────────────────────┼───────────────────────┤
- │ FAIL TO REJECT H₀ │ CORRECT DECISION │ TYPE II ERROR (β) │
- │ (Conclude No Difference) │ (1 - α) │ "False Negative" │
- └───────────────────────────────┴───────────────────────┴───────────────────────┘
- 1. The Hypotheses
Null Hypothesis ( H 0 H_0 H 0 ): There is no true difference between the experimental treatment and control ( R R = 1.0 RR = 1.0 RR = 1.0 , A R R = 0 ARR = 0 A RR = 0 ).
Alternative Hypothesis ( H 1 H_1 H 1 ): A true difference exists between the treatments ( R R ≠ 1.0 RR \neq 1.0 RR = 1.0 , A R R ≠ 0 ARR \neq 0 A RR = 0 ).
2. Type I Error ( α \alpha α ) — "False Positive"
Occurs when researchers reject the null hypothesis when H 0 H_0 H 0 is actually true, falsely concluding that an ineffective drug is effective. In biomedical research, the alpha threshold is conventionally set at α = 0.05 \alpha = 0.05 α = 0.05 ( 5 % 5\% 5% ), meaning researchers accept a maximum 5 % 5\% 5% probability that a positive finding occurred purely by chance.
3. Type II Error ( β \beta β ) — "False Negative"
Occurs when researchers fail to reject the null hypothesis when H 0 H_0 H 0 is actually false, missing a true therapeutic effect. Beta is conventionally set between β = 0.10 \beta = 0.10 β = 0.10 and 0.20 0.20 0.20 ( 10 % 10\% 10% to 20 % 20\% 20% ).
4. Statistical Power ( 1 − β 1 - \beta 1 − β )
Statistical power is the probability that a trial will detect a statistically significant difference if a true difference of a given magnitude actually exists. Standard clinical trial design demands power of at least 80 % 80\% 80% ( 1 − 0.20 1 - 0.20 1 − 0.20 ) or 90 % 90\% 90% ( 1 − 0.10 1 - 0.10 1 − 0.10 ).
Key Determinants of Statistical Power:
Sample Size ( N N N ): Larger sample size increases power.
Effect Size: Detecting a large therapeutic difference requires fewer patients than detecting a subtle difference.
Baseline Event Rate: In trials with rare clinical events, total event count—not just patient count—dictates power. If fewer events occur than projected, power plummets.
Significance Level ( α \alpha α ): A more stringent alpha (e.g., 0.01 0.01 0.01 vs. 0.05 0.05 0.05 ) reduces power.
5. The p p p -value: Definition and Misconceptions
The p p p -value is the probability of obtaining an effect equal to or more extreme than the observed trial result, assuming that the null hypothesis is true.
If p < 0.05 p < 0.05 p < 0.05 : The result is deemed statistically significant , and H 0 H_0 H 0 is rejected.
Crucial Clinical Truths:
A p p p -value does not indicate the probability that the hypothesis is true.
A p p p -value does not reflect the clinical magnitude or importance of the effect. With massive sample sizes ( N > 50 , 000 N > 50,000 N > 50 , 000 ), a clinically trivial difference (e.g., systolic blood pressure reduction of 0.4 mmHg 0.4\text{ mmHg} 0.4 mmHg ) can achieve p < 0.001 p < 0.001 p < 0.001 .
A p p p -value ≥ 0.05 \ge 0.05 ≥ 0.05 does not prove that treatments are equivalent; it merely indicates that the study lacked sufficient evidence to reject the null hypothesis ("absence of evidence is not evidence of absence").
Confidence Intervals: Interpretation of Difference vs. Ratio Metrics
A 95% Confidence Interval ( 95 % CI 95\%\text{ CI} 95% CI ) represents the range of plausible values within which the true population effect parameter lies with 95 % 95\% 95% certainty. Confidence intervals convey both statistical significance and the precision of the estimate (narrow intervals reflect high precision; wide intervals indicate low precision due to small sample size or sparse events). HOW TO INTERPRET A 95% CONFIDENCE INTERVAL
- DIFFERENCE METRICS (ARR, Mean Difference) RATIO METRICS (RR, OR, HR)
- ───────────────────────────────────────── ──────────────────────────
- Null Value = 0 Null Value = 1.0
- CI: [0.02 to 0.08] CI: [0.65 to 0.88]
- ────●──────●──────► ────●──────●──────►
- 0 0 1.0
- Excludes 0 ──► STATISTICALLY SIGNIFICANT Excludes 1.0 ──► STATISTICALLY SIGNIFICANT
- (p < 0.05) (p < 0.05)
- CI: [-0.01 to 0.06] CI: [0.82 to 1.15]
- ────●───0──●──────► ────●───1.0──●────►
- ▲ ▲
- Crosses 0 ──► NOT STATISTICALLY SIGNIFICANT Crosses 1.0 ──► NOT STATISTICALLY SIGNIFICANT
- (p ≥ 0.05) (p ≥ 0.05)
Difference Metrics vs. Ratio Metrics Summary:
Difference Metrics ( A R R ARR A RR , Absolute Risk Increase, Mean Difference in Blood Pressure or HbA1c):
Null Value is 0 0 0 .
Statistically significant ( p < 0.05 p < 0.05 p < 0.05 ) if the 95 % CI 95\%\text{ CI} 95% CI excludes 0 0 0 (both boundaries are positive or both are negative).
Not statistically significant ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ) if the 95 % CI 95\%\text{ CI} 95% CI crosses 0 0 0 (lower bound is negative, upper bound is positive).
Ratio Metrics ( R R RR RR , Odds Ratio, Hazard Ratio):
Null Value is 1.0 1.0 1.0 .
Statistically significant ( p < 0.05 p < 0.05 p < 0.05 ) if the 95 % CI 95\%\text{ CI} 95% CI excludes 1.0 1.0 1.0 (both boundaries < 1.0 < 1.0 < 1.0 or both > 1.0 > 1.0 > 1.0 ).
Not statistically significant ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ) if the 95 % CI 95\%\text{ CI} 95% CI crosses 1.0 1.0 1.0 (e.g., 0.85 0.85 0.85 to 1.20 1.20 1.20 ).
Clinical Significance vs. Statistical Significance
A statistically significant finding ( p < 0.05 p < 0.05 p < 0.05 , CI excluding null) does not guarantee that the intervention is clinically meaningful. Clinicians must compare the confidence interval bounds against the Minimal Clinically Important Difference (MCID) —the smallest change in outcome that patients or clinicians perceive as beneficial. If a confidence interval falls entirely below the MCID, the drug's effect is clinically trivial despite statistical significance.