Study guide
13.2 Hypothesis Testing, Power, Type I/II Errors, and Confidence Intervals
EDAIC Part I European Diploma in Anaesthesiology and Intensive Care
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: The EDAIC Part I Examination
1.1 EDAIC Part I Exam Format, Scoring and Multiple True/False Strategy
2 Chapter 2: Anatomy for Anaesthesia and Intensive Care
2.1 Functional Anatomy of the Upper Airway, Larynx, and Tracheobronchial Tree 2.2 Neuraxial Anatomy: Spine, Spinal Cord, Meninges, and Spaces 2.3 Vascular Anatomy for Central and Peripheral Cannulation 2.4 Peripheral Nerve Plexuses and Autonomic Nervous System Pathways 2.5 Thoracic Anatomy: Chest Wall, Intercostal Spaces, Pleura, Mediastinum and Heart
3 Chapter 3: Respiratory and Cardiovascular Physiology
3.1 Respiratory Mechanics, Lung Volumes, and Work of Breathing 3.2 Ventilation-Perfusion Relationships, Shunt, and Gas Exchange 3.3 Control of Breathing, Hypoxia, Altitude and Hyperbaric Physiology 3.4 Cardiac Cycle, Electrophysiology, and Ventricular Function 3.5 Vascular Resistance, Microcirculation, and Systemic Hemodynamics
4 Chapter 4: Neurophysiology, Renal, Hepatic and Acid-Base Physiology
4.1 Cerebral Blood Flow, Intracranial Pressure, and CSF Dynamics 4.2 Renal Physiology, Glomerular Filtration, and Tubular Transport 4.3 Hepatic Blood Flow, Metabolism, and Clearance 4.4 Acid-Base Regulation, Buffering Systems, and Stewart Approach
5 Chapter 5: Endocrine and Haematological Physiology
5.1 Endocrine Physiology and the Neuroendocrine Stress Response 5.2 Haemostasis Physiology, Coagulation Testing and Inherited Bleeding Disorders
6 Chapter 6: Special Patient Physiology: Maternal, Paediatric and Geriatric
6.1 Maternal Physiological Adaptations to Pregnancy and Uteroplacental Flow 6.2 Neonatal and Paediatric Physiological Differences 6.3 Physiological Changes of Ageing and Thermoregulation
7 Chapter 7: General Pharmacological Principles and Inhalational Anaesthetics
7.1 Pharmacokinetic Principles, Compartmental Models, and Clearance 7.2 Pharmacodynamic Mechanisms, Receptor Theory, and Drug Interactions 7.3 Physical Properties, Uptake, and Distribution of Volatile Anaesthetics 7.4 Minimum Alveolar Concentration (MAC), Organ Effects, and Toxicity
8 Chapter 8: Intravenous Agents, Neuromuscular Blockade and Analgesics
8.1 Intravenous Induction Agents: Propofol, Etomidate, Ketamine, and Barbiturates 8.2 Neuromuscular Blocking Agents: Depolarizing and Non-Depolarizing Drugs 8.3 Neuromuscular Monitoring and Reversal: Anticholinesterases and Sugammadex 8.4 Opioid Pharmacology, Receptors, and Non-Opioid Analgesic Agents
9 Chapter 9: Cardiovascular Drugs, Autonomic Pharmacology and Local Anaesthetics
9.1 Inotropes, Vasopressors, and Inodilators in Perioperative Care 9.2 Antiarrhythmic, Antihypertensive, and Beta-Adrenergic Antagonist Drugs 9.3 Local Anaesthetic Pharmacology, Toxicity (LAST), and Lipid Emulsion Therapy
10 Chapter 10: Haematological and Internal Medicine Pharmacology
10.1 Anticoagulant, Antiplatelet and Haemostatic Drugs and Their Reversal 10.2 Internal Medicine Drugs: Endocrine, Respiratory, Psychiatric and Antiepileptic Agents and Their Anaesthetic Interactions
11 Chapter 11: Physics, Clinical Measurement and Medical Equipment
11.1 Physical Gas Laws, Fluid Dynamics, and Resistance Principles 11.2 Anaesthetic Machines, Gas Supply Pipelines, Cylinders, and Safety Mechanisms 11.3 Breathing Systems, Vaporizer Mechanics, and Scavenging Systems 11.4 Invasive and Non-Invasive Hemodynamic and Respiratory Monitoring
12 Chapter 12: Electrical Safety, Temperature, Humidity and Gas Analysis
12.1 Electricity, Electrical Safety, Diathermy, Defibrillation, Implanted Devices and Fire 12.2 Temperature Measurement, Humidity, Gas Analysis and SI Units
13 Chapter 13: Biostatistics, Clinical Trials and Evidence Appraisal
13.1 Descriptive Statistics, Distributions, and Parametric vs Non-Parametric Tests 13.2 Hypothesis Testing, Power, Type I/II Errors, and Confidence Intervals 13.3 Clinical Study Designs, Bias, and Diagnostic Test Performance (Sensitivity, Specificity, ROC)
14 Chapter 14: General Anaesthesia, Preoperative Evaluation and Airway Management
14.1 Preoperative Risk Assessment, Cardiovascular Risk Indices, and Fasting Guidelines 14.2 The Difficult Airway: Assessment, Algorithms, and Rescue Oxygenation 14.3 Intraoperative Fluid Therapy, Goal-Directed Hemodynamics, and Blood Conservation 14.4 Post-Anaesthesia Care Unit (PACU) Management, Emergence, and Postoperative Nausea/Vomiting
15 Chapter 15: Coexisting Disease and Preoperative Investigations
15.1 Interpreting Investigations: ECG, Chest Radiograph, Lung Function and Blood Tests 15.2 Coexisting Disease I: Asthma, COPD, Obstructive Sleep Apnoea, Obesity, Liver and Renal Disease 15.3 Endocrine Disease and Emergencies: Diabetes, DKA and HHS, Thyroid, Adrenal, Phaeochromocytoma and Carcinoid 15.4 Coexisting Disease II: Neuromuscular and Neurological Disease
16 Chapter 16: Subspecialty Anaesthesia: Cardiothoracic, Vascular and Neuroanaesthesia
16.1 Anaesthesia for Cardiac Surgery and Cardiopulmonary Bypass Management 16.2 Thoracic Anaesthesia, One-Lung Ventilation, and Major Vascular Surgery 16.3 Neuroanaesthesia: Craniotomy, Sitting Position, and Traumatic Brain Injury
17 Chapter 17: Obstetric, Paediatric and Ambulatory Anaesthesia
17.1 Obstetric Analgesia and Anaesthesia: Labour, Caesarean Delivery, and High-Risk Conditions 17.2 Paediatric Anaesthesia: Induction, Airway Management, and Emergencies 17.3 Ambulatory Day-Surgery Selection, Premedication, and Elderly Patient Anaesthesia
18 Chapter 18: Anaesthetic Complications and Specialty Settings
18.1 Anaesthetic Complications: Accidental Awareness, Aspiration, Positioning Injuries, Visual Loss and Dental Damage 18.2 Anaesthesia for ENT and Laser Airway Surgery, Ophthalmic Surgery, and Laparoscopic and Robotic Surgery
19 Chapter 19: Regional Anaesthesia and Acute/Chronic Pain Management
19.1 Neuraxial Techniques: Spinal, Epidural, and Caudal Anaesthesia 19.2 Upper and Lower Limb Peripheral Nerve and Fascial Plane Blocks 19.3 Multimodal Acute Postoperative Pain Management and ERAS Pathways 19.4 Chronic and Neuropathic Pain Syndromes and Interventional Management 19.5 Palliative Symptom Control, Psychological Aspects of Pain and Multidisciplinary Pain Management
20 Chapter 20: Intensive Care Medicine: Ventilation, Shock and Organ Support
20.1 Acute Respiratory Distress Syndrome (ARDS) and Mechanical Ventilation Strategies 20.2 Sepsis-3 Definitions, Septic Shock, and Hemodynamic Resuscitation 20.3 Acute Kidney Injury, Indications for Dialysis, and Continuous Renal Replacement Therapy 20.4 ICU Sedation, Delirium, Weaning, and Brain Death Determination
21 Chapter 21: Intensive Care Medicine: Admission, Nutrition, Infection and Electrolytes
21.1 ICU Admission, Severity Scoring, Nutrition and Routine Prophylaxis 21.2 Antimicrobial Therapy, Stewardship and ICU-Acquired Infections 21.3 Electrolyte Disorders: Sodium, Potassium, Calcium, Magnesium and Phosphate
22 Chapter 22: Intensive Care and Emergency Medicine: Neurocritical Care, Poisoning and Cardiorespiratory Emergencies
22.1 Neurocritical Care: Subarachnoid Haemorrhage, Stroke, Status Epilepticus and Acute Neuromuscular Failure 22.2 Acute Poisoning: Toxidromes, Decontamination and Specific Antidotes 22.3 Acute Cardiorespiratory Emergencies: Pulmonary Embolism, Acute Severe Asthma and COPD, Non-Invasive Support and Acute Heart Failure
23 Chapter 23: Resuscitation, Perioperative Emergencies and Professional Ethics
23.1 Advanced Life Support (ALS) Guidelines and Perioperative Cardiac Arrest 23.2 Perioperative Crises: Malignant Hyperthermia and Severe Anaphylaxis 23.3 Major Trauma, Hemorrhagic Shock, and Massive Transfusion Protocols 23.4 Professional Ethics, Informed Consent, Patient Safety, and Helsinki Declaration
24 Chapter 24: Newborn, Paediatric and Obstetric Emergencies
24.1 Newborn Life Support and Paediatric Life Support 24.2 Obstetric Emergencies: Maternal Collapse, Amniotic Fluid Embolism, Uterine Inversion and Resuscitative Hysterotomy
25 Chapter 25: Burns, Environmental Emergencies and Patient Transport
25.1 Burns, Inhalation Injury, Drowning, Hypothermia, Heat Illness and Decompression Illness 25.2 Prehospital Care and Intra- and Inter-Hospital Transport of the Critically Ill
13.2 Hypothesis Testing, Power, Type I/II Errors, and Confidence Intervals
Key Takeaways Explain This Topic AI
The null hypothesis ( H 0 H_0 H 0 ) states that there is no true difference or association between study groups; the P-value is the probability of observing a test statistic as extreme as or more extreme than the observed data, assuming H 0 H_0 H 0 is true.
A Type I error (probability α \alpha α , typically 0.05 0.05 0.05 ) occurs when a true null hypothesis is falsely rejected; a Type II error (probability β \beta β , typically 0.10 0.10 0.10 to 0.20 0.20 0.20 ) occurs when a false null hypothesis is mistakenly retained.
Statistical power ( 1 − β 1 - \beta 1 − β , target ≥ 80 % \ge 80\% ≥ 80% ) is the probability of correctly rejecting a false null hypothesis; it is augmented by larger sample size ( n n n ), greater anticipated effect size ( Δ \Delta Δ ), higher alpha threshold ( α \alpha α ), and lower population variance ( σ 2 \sigma^2 σ 2 ).
A 95 % 95\% 95% confidence interval is a range of values compatible with the data: if a study were repeated many times, 95 % 95\% 95% of such intervals would contain the true population parameter; for differences between means or proportions, an interval crossing 0 0 0 denotes non-significance ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ), whereas for ratio metrics (RR, OR, HR), an interval spanning 1.0 1.0 1.0 indicates lack of statistical significance.
Statistical significance does not equal clinical significance: very large trials can yield minute, clinically meaningless differences with p < 0.001 p < 0.001 p < 0.001 , while underpowered trials may fail to achieve statistical significance ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ) despite a large, life-saving clinical treatment effect.
Last updated: October 2026
13.2 Hypothesis Testing, Power, Type I/II Errors, and Confidence Intervals
Inferential statistics enables clinicians to draw valid scientific conclusions about broad patient populations using finite clinical study samples. Mastery of hypothesis testing, statistical error types, sample size determination, and confidence interval estimation is indispensable for independent critical appraisal.
1. The Inferential Framework: Null and Alternative Hypotheses
Modern biomedical research relies on the hypothetico-deductive paradigm established by Karl Popper and formalized statistically by Ronald Fisher, Jerzy Neyman, and Egon Pearson. In this framework, empirical science cannot prove a proposition conclusively true; it can only attempt to falsify a pre-specified null hypothesis.
The Null Hypothesis ( H 0 H_0 H 0 )
Definition: The baseline proposition that there is no true difference , treatment effect, or association between the examined study interventions in the target population.
Mathematical Formulation: For two continuous means, H 0 : μ 1 − μ 2 = 0 H_0: \mu_1 - \mu_2 = 0 H 0 : μ 1 − μ 2 = 0 ; for binary event ratios, H 0 : R R = 1.0 H_0: RR = 1.0 H 0 : RR = 1.0 or O R = 1.0 OR = 1.0 OR = 1.0 .
All statistical test calculations begin by assuming that H 0 H_0 H 0 is absolute truth.
The Alternative Hypothesis ( H 1 H_1 H 1 or H A H_A H A )
Definition: The proposition that a genuine treatment effect, difference, or association exists between study groups.
Two-Tailed (Bilateral) Hypothesis: Postulates that the groups differ without pre-specifying directionality ( H 1 : μ 1 ≠ μ 2 H_1: \mu_1 \neq \mu_2 H 1 : μ 1 = μ 2 ). This is the scientific gold standard in clinical anaesthesia trials because an intervention may prove unexpectedly superior, neutral, or harmful.
One-Tailed (Unilateral) Hypothesis: Postulates an effect in a single pre-specified direction ( H 1 : μ 1 > μ 2 H_1: \mu_1 > \mu_2 H 1 : μ 1 > μ 2 or H 1 : μ 1 < μ 2 H_1: \mu_1 < \mu_2 H 1 : μ 1 < μ 2 ). One-tailed tests double the nominal significance level in the chosen tail, making statistical significance easier to achieve. However, they completely ignore the possibility of harm and are rarely acceptable in regulatory or clinical trials unless a unidirectional effect is physiologically mandated.
- 2. The P-value: Mathematical Definition and Clinical Fallacies
- Rigorous Definition
The P-value is the conditional probability of obtaining a test statistic at least as extreme as, or more extreme than, the value observed in the study data, calculated under the assumption that the null hypothesis ( H 0 H_0 H 0 ) is strictly true : P = P ( Data as extreme or more extreme ∣ H 0 is true ) P = P(\text{Data as extreme or more extreme} \mid H_0\text{ is true}) P = P ( Data as extreme or more extreme ∣ H 0 is true )
If the calculated P-value falls below a pre-determined significance threshold ( α \alpha α , conventionally 0.05 0.05 0.05 ), the observed difference is deemed unlikely to have arisen solely through random sampling variation, and H 0 H_0 H 0 is rejected.
The Five P-value Fallacies Tested in Examinations
The Fallacy of the Null Probability: The P-value is NOT the probability that the null hypothesis is true. P ( Data ∣ H 0 ) ≠ P ( H 0 ∣ Data ) P(\text{Data} \mid H_0) \neq P(H_0 \mid \text{Data}) P ( Data ∣ H 0 ) = P ( H 0 ∣ Data ) . Calculating P ( H 0 ∣ Data ) P(H_0 \mid \text{Data}) P ( H 0 ∣ Data ) requires Bayesian inference incorporating prior probability.
The Fallacy of Chance: A P-value of 0.04 0.04 0.04 does NOT mean there is a 4 % 4\% 4% probability that the study results occurred by chance. The null hypothesis was already assumed to be true during test calculation.
The Fallacy of Effect Magnitude: A very small P-value (e.g., p < 0.0001 p < 0.0001 p < 0.0001 ) does NOT indicate a large or clinically meaningful treatment effect. In huge cohorts, trivial physiological differences achieve extreme statistical significance.
The Fallacy of Equivalence ("Absence of Evidence"): A non-significant P-value ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ) does NOT prove that the treatments are equivalent or that the null hypothesis is true. It merely indicates that the study failed to detect sufficient evidence to reject H 0 H_0 H 0 , frequently due to inadequate sample size.
The Fallacy of Replicability: 1 − P 1 - P 1 − P does NOT represent the probability that a repeat trial will achieve statistical significance.
3. Statistical Error Typology: Type I ( α \alpha α ) vs Type II ( β \beta β ) Errors
When testing a statistical hypothesis, the investigator's decision to reject or retain H 0 H_0 H 0 can intersect with biological reality in four distinct ways. BIOLOGICAL TRUTH
- H0 True H0 False
- (No True Effect) (True Effect Exists)
- +-------------------+-------------------+
- Reject H0 | TYPE I ERROR | CORRECT DECISION |
- (Claim Effect) | (Alpha, α) | Statistical Power |
- DECISION | False Positive | (1 - Beta) |
- +-------------------+-------------------+
- Fail to Reject | CORRECT DECISION | TYPE II ERROR |
- H0 | Confidence | (Beta, β) |
- (No Effect) | (1 - Alpha) | False Negative |
- +-------------------+-------------------+
- Type I Error ( α \alpha α , False Positive)
Definition: Rejecting the null hypothesis when H 0 H_0 H 0 is actually true in reality; concluding that a treatment effect exists when there is no genuine biological difference.
Significance Level ( α \alpha α ): The maximum acceptable probability of committing a Type I error, set a priori by the researcher (conventionally α = 0.05 \alpha = 0.05 α = 0.05 or 5 % 5\% 5% ).
Clinical Consequence: Adopting an ineffective, costly, or potentially toxic drug into clinical practice under the false impression of therapeutic superiority.
Type II Error ( β \beta β , False Negative)
Definition: Failing to reject the null hypothesis when H 0 H_0 H 0 is actually false in reality; failing to detect a genuine clinical treatment effect that truly exists.
Type II Error Rate ( β \beta β ): The probability of committing a Type II error, standardly targeted at β = 0.10 \beta = 0.10 β = 0.10 ( 10 % 10\% 10% ) or β = 0.20 \beta = 0.20 β = 0.20 ( 20 % 20\% 20% ).
Clinical Consequence: Discarding or abandoning a truly effective, life-saving therapeutic agent because a trial had inadequate statistical power to prove its efficacy.
The Inverse Trade-off
For a fixed sample size, α \alpha α and β \beta β are inversely linked. Lowering α \alpha α (e.g., from 0.05 0.05 0.05 to 0.01 0.01 0.01 ) reduces the false-positive risk but inevitably increases β \beta β (the false-negative risk), thereby suppressing statistical power. The only mechanism to simultaneously reduce both α \alpha α and β \beta β is to increase sample size ( n n n ).
- 4. Statistical Power and Sample Size Calculation
- Statistical Power ( 1 − β 1 - \beta 1 − β )
Statistical power is the probability of correctly rejecting a false null hypothesis when a true treatment effect of a specified magnitude exists in the population: Power = 1 − β \text{Power} = 1 - \beta Power = 1 − β
In clinical research, the standard minimum acceptable power is 80 % 80\% 80% ( 0.80 0.80 0.80 ) , with higher-stakes trials powered to 90 % 90\% 90% ( 0.90 0.90 0.90 ) .
The Four Interdependent Pillars of Sample Size Determination
To calculate the sample size ( n n n ) required for a trial, four variables must be specified. Modifying any one variable alters the remaining parameters:
Significance Level ( α \alpha α ): The false-positive threshold. Setting a stricter alpha (e.g., 0.01 0.01 0.01 instead of 0.05 0.05 0.05 ) requires a larger sample size.
Statistical Power ( 1 − β 1 - \beta 1 − β ): Targeting higher power (e.g., 90 % 90\% 90% instead of 80 % 80\% 80% ) requires a larger sample size.
Anticipated Effect Size ( Δ \Delta Δ or δ = μ 1 − μ 2 \delta = \mu_1 - \mu_2 δ = μ 1 − μ 2 ): The magnitude of difference the trial is designed to detect. Detecting subtle, small differences requires substantially larger cohorts than detecting dramatic clinical effects ( n ∝ 1 / Δ 2 n \propto 1 / \Delta^2 n ∝ 1/ Δ 2 ).
Population Variance ( σ 2 \sigma^2 σ 2 or S D SD S D ): The background biological scatter. Greater variance obscures the treatment signal, demanding a larger sample size ( n ∝ σ 2 n \propto \sigma^2 n ∝ σ 2 ).
Simplified Sample Size Formula (Continuous Means)
For a two-arm trial with equal allocation ( 1 : 1 1:1 1 : 1 ) comparing two independent continuous means with variance σ 2 \sigma^2 σ 2 : n = 2 ( Z α / 2 + Z β ) 2 ⋅ σ 2 Δ 2 n = \frac{2 \left( Z_{\alpha/2} + Z_\beta \right)^2 \cdot \sigma^2}{\Delta^2} n = Δ 2 2 ( Z α /2 + Z β ) 2 ⋅ σ 2
Where:
For a two-tailed α = 0.05 \alpha = 0.05 α = 0.05 , Z α / 2 = 1.96 Z_{\alpha/2} = 1.96 Z α /2 = 1.96 .
For 80 % 80\% 80% power ( β = 0.20 \beta = 0.20 β = 0.20 ), Z β = 0.84 Z_\beta = 0.84 Z β = 0.84 .
For 90 % 90\% 90% power ( β = 0.10 \beta = 0.10 β = 0.10 ), Z β = 1.28 Z_\beta = 1.28 Z β = 1.28 .
Multiplicity and Bonferroni Correction: When a trial evaluates multiple primary endpoints or performs repeated interim looks, the cumulative probability of at least one Type I error escalates rapidly ( 1 − ( 1 − α ) k 1 - (1 - \alpha)^k 1 − ( 1 − α ) k ). To preserve an overall family-wise error rate of 0.05 0.05 0.05 across k k k comparisons, the Bonferroni correction sets the significance threshold for each individual test to α ∗ = α / k \alpha^* = \alpha / k α ∗ = α / k .
5. Confidence Intervals (CI): Derivation and Null Value Rules
While a P-value provides a binary test of hypothesis against an arbitrary cut-point, a Confidence Interval (CI) conveys both the point estimate of effect size and the degree of measurement precision.
Mathematical Formulation
For normally distributed continuous data, a 95 % 95\% 95% confidence interval around a sample mean is calculated as: 95 % CI = x ˉ ± ( 1.96 × S E M ) = x ˉ ± 1.96 × S D n 95\%\text{ CI} = \bar{x} \pm (1.96 \times SEM) = \bar{x} \pm 1.96 \times \frac{SD}{\sqrt{n}} 95% CI = x ˉ ± ( 1.96 × SEM ) = x ˉ ± 1.96 × n S D
For a 99 % 99\% 99% confidence interval, the critical multiplier expands to 2.58 2.58 2.58 .
Frequentist Interpretation
The correct frequentist interpretation is: If the study were repeated an infinite number of times under identical experimental conditions, 95 % 95\% 95% of the calculated 95 % 95\% 95% confidence intervals would contain the true, unknown population parameter. It does not mean that there is a 95 % 95\% 95% probability that the true mean lies within the specific numbers of a single published interval (in classical frequentist statistics, the true parameter is a fixed constant, not a random variable).
The Null Hypothesis Equivalence Rules
A confidence interval serves as an immediate test of statistical significance at the corresponding α \alpha α level (a 95 % CI 95\%\text{ CI} 95% CI corresponds to α = 0.05 \alpha = 0.05 α = 0.05 ): DIFFERENCE METRICS (Null = 0)
- Statistically Significant Non-Significant
- [-----] [------0------]
- <----+------+------+------+----> <----+------+------+------+---->
- 2 -1 0 +1 -2 -1 0 +1
- (Excludes 0) (Crosses 0)
- RATIO METRICS (Null = 1.0)
- Statistically Significant Non-Significant
- [-----] [------1.0------]
- <----+------+------+------+----> <----+------+------+------+---->
- 0.2 0.6 1.0 1.4 0.2 0.6 1.0 1.4
- (Excludes 1.0) (Crosses 1.0)
Additive / Difference Measures (Mean Difference, Risk Difference, ARR):
The null value of no effect is 0 0 0 .
If the 95 % CI 95\%\text{ CI} 95% CI excludes 0 0 0 (e.g., mean difference in extubation time − 12.4 min -12.4\text{ min} − 12.4 min to − 3.2 min -3.2\text{ min} − 3.2 min ), the finding is statistically significant ( p < 0.05 p < 0.05 p < 0.05 ).
If the 95 % CI 95\%\text{ CI} 95% CI includes 0 0 0 (e.g., − 2.1 min -2.1\text{ min} − 2.1 min to + 4.5 min +4.5\text{ min} + 4.5 min ), the finding is not statistically significant ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ).
Multiplicative / Ratio Measures (Relative Risk, Odds Ratio, Hazard Ratio):
The null value of no effect is 1.0 1.0 1.0 .
If the 95 % CI 95\%\text{ CI} 95% CI excludes 1.0 1.0 1.0 (e.g., R R = 0.65 RR = 0.65 RR = 0.65 , 95 % CI 0.48 − 0.88 95\%\text{ CI } 0.48 - 0.88 95% CI 0.48 − 0.88 ), the finding is statistically significant ( p < 0.05 p < 0.05 p < 0.05 ).
If the 95 % CI 95\%\text{ CI} 95% CI spans or touches 1.0 1.0 1.0 (e.g., O R = 1.15 OR = 1.15 OR = 1.15 , 95 % CI 0.92 − 1.44 95\%\text{ CI } 0.92 - 1.44 95% CI 0.92 − 1.44 ), the finding is not statistically significant ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ).
6. Statistical Significance versus Clinical Relevance
A critical objective of the EDAIC examination is distinguishing mathematical significance from bedside clinical value.
The Overpowered Trial Trap
When a study enrolls tens of thousands of participants, standard errors shrink toward zero ( S E M ∝ 1 / n SEM \propto 1/\sqrt{n} SEM ∝ 1/ n ). Consequently, clinically imperceptible physiological differences achieve extreme statistical significance.
Example: An ultra-large multicentre trial ( n = 40,000 n = 40{,}000 n = 40 , 000 ) evaluates a new vasopressor and finds a mean arterial pressure increase of 1.1 mmHg 1.1\text{ mmHg} 1.1 mmHg ( p = 0.0002 p = 0.0002 p = 0.0002 ). While statistically irrefutable, an elevation of 1.1 mmHg 1.1\text{ mmHg} 1.1 mmHg is clinically irrelevant at the bedside and does not justify the cost or adverse risk of changing practice.
The Underpowered Trial Trap (Type II Error)
Small clinical studies frequently fail to achieve statistical significance ( p ≥ 0.05 p \ge 0.05 p ≥ 0.05 ) despite a large, life-saving clinical treatment effect.
Example: A pilot study ( n = 20 n = 20 n = 20 per group) of a new antiemetic reduces postoperative vomiting from 40 % 40\% 40% to 20 % 20\% 20% . Because the sample size was tiny, the chi-squared test yields p = 0.16 p = 0.16 p = 0.16 . Concluding that "the antiemetic has no effect" is a severe error of interpretation; the trial was simply underpowered to reject H 0 H_0 H 0 .
The Minimally Clinically Important Difference (MCID)
The MCID is the smallest change in an outcome measure that clinicians and patients perceive as clinically beneficial and that would mandate an active change in clinical management. Clinical trials must be powered specifically to detect the MCID rather than an arbitrarily chosen mathematical difference.