Study guide
7.3 Validity, Reliability (ICC, CV%, TE), and Smallest Worthwhile Change
NSCA Certified Performance and Sport Scientist CPSS
Track your progress as you read
Finished sections get a check mark.
1 Introduction: CPSS Exam Overview, Eligibility, and Case-Study Strategy
1.1 CPSS Exam Format, Eligibility Pathways, and the 2026 Blueprint 1.2 Mastering Athlete Case Studies and Research Review Scenarios
2 Chapter 1: Scientific Process, Adaptation Theory, and Periodization (Domain 1)
2.1 Training Principles, Adaptation Models, and Injury-Risk Mitigation 2.2 Evaluating Performance Programs Using the Scientific Process 2.3 Periodization Paradigms, Fitness-Fatigue Models, and Tapering
3 Chapter 2: Scientific Disciplines in Athlete Preparation (Domain 1)
3.1 Exercise Physiology, Bioenergetics, and Systemic Adaptations 3.2 Applied Biomechanics, Force-Time Kinetics, and Force-Velocity Profiling 3.3 Motor Learning, Skill Acquisition, and Constraints-Led Practice Design 3.4 Performance Nutrition, Recovery Modalities, and Sport Psychology
4 Chapter 3: Technology Review, Research Appraisal, and Knowledge Translation (Domain 1)
4.1 Critically Reviewing Sport Performance Products, Technologies, and Trends 4.2 Evaluating Research Studies: Design, Validity, and Practical Relevance 4.3 Communicating Research Findings to Support Evidence-Based Practice
5 Chapter 4: Sport Demands Analysis, KPIs, and Resource Strategy (Domain 2)
5.1 Researching Biomechanical, Tactical, and Technical Sport Demands 5.2 Physiological Demands, Injury Epidemiology, and Environmental Constraints 5.3 Qualitative and Mixed-Methods Research for Sport Demands Analysis 5.4 Collaborating with Stakeholders to Establish Key Performance Indicators 5.5 Identifying Resources and Strategies for Short- and Long-Term Goals
6 Chapter 5: Profiling Benchmarks and Needs Analysis Communication (Domain 2)
6.1 Developing Normative Benchmarks: Z-Scores, T-Scores, and Percentiles 6.2 Multidimensional Athlete Profiling and Return-to-Performance Milestones 6.3 Communicating Needs Analysis Results to Multidisciplinary Stakeholders
7 Chapter 6: Assessment Ethics, Safety Standards, and Tool Psychometrics (Domain 3)
7.1 Ethical, Legal, and Privacy Considerations in Athlete Data Collection 7.2 Evaluating Assessment Environments, Equipment, and Safety Standards 7.3 Validity, Reliability (ICC, CV%, TE), and Smallest Worthwhile Change
8 Chapter 7: Applied Assessment and Load-Monitoring Systems (Domain 3)
8.1 Movement Tracking Systems: GNSS/GPS, LPS, IMUs, and Optical Tracking 8.2 Neuromuscular Testing: Force Platforms, Isometric Pulls/Pushes, and VBT 8.3 Internal Load, Autonomic Readiness (HRV), Biomarkers, and PROMs
9 Chapter 8: Quality Assurance, Protocols, and Assessment Efficacy (Domain 3)
9.1 Applying Quality Assurance Techniques in Sport Science Data Collection 9.2 Developing, Documenting, and Standardizing Data Collection Protocols 9.3 Systematically Reevaluating Assessments, Protocols, and Interventions
10 Chapter 9: Data Governance, Storage Architecture, and Cleaning (Domain 4)
10.1 Data Privacy, Security Protocols, and Protecting Organizational Information 10.2 Designing Plans for Data Collection, Storage, Analysis, and AMS Workflows 10.3 Evaluating Data Quality, Signal Integrity, Outliers, and Cleaning Procedures
11 Chapter 10: Statistical Modeling, Feedback Delivery, and Analytical Limitations (Domain 4)
11.1 Inferential Statistics, Correlation, Regression, ANOVA, and Effect Sizes 11.2 Longitudinal Load Modeling: ACWR, EWMA, Monotony, Strain, and Change Detection 11.3 Data Visualization, Dashboard Design, and Timely Stakeholder Feedback 11.4 Communicating Analytical Limitations, Bias, and Uncertainty
7.3 Validity, Reliability (ICC, CV%, TE), and Smallest Worthwhile Change
Key Takeaways Explain This Topic AI
Concurrent criterion validity requires evaluating systematic bias (d-bar) and 95% Limits of Agreement (d-bar +/- 1.96 * SD_d) via Bland-Altman analysis alongside Standard Error of the Estimate (SEE), because Pearson r alone cannot detect constant systematic offset
Relative reliability is quantified by the Intraclass Correlation Coefficient (ICC), which reflects rank-order stability within a group but is highly sensitive to sample heterogeneity (a homogeneous elite squad can depress ICC even when absolute trial-to-trial error is low)
Absolute reliability is quantified in raw physical units via Typical Error of Measurement (TE = SD_diff / sqrt(2)) and as a dimensionless percentage via Coefficient of Variation (CV% = [TE / Mean] * 100%), with standard sport science benchmarks requiring CV% <= 5% and ICC >= 0.80–0.90
The Smallest Worthwhile Change (SWC = 0.2 * SD_between) defines the minimum practically meaningful signal, while the Minimal Detectable Change at 95% confidence (MDC_95 = 1.96 * sqrt(2) * TE = 2.77 * TE) defines statistical certainty above measurement noise
A test demonstrates 'Good' sensitivity to individual change when its noise is smaller than the signal (TE < SWC), and an athlete's observed change (Delta) is interpreted as real and actionable when it exceeds both TE (or the chosen TE multiple) and SWC
Last updated: October 2026
Validity, Reliability (ICC, CV%, TE), and Smallest Worthwhile Change
Quick Answer: Validity confirms that a tool measures what it claims to measure (evaluated against a gold standard via Standard Error of the Estimate [ SEE \text{SEE} SEE ] and Bland-Altman 95 % 95\% 95% Limits of Agreement [ d ˉ ± 1.96 × SD d \bar{d} \pm 1.96 \times \text{SD}_{d} d ˉ ± 1.96 × SD d ] ). Reliability quantifies trial-to-trial reproducibility: Relative reliability ( ICC \text{ICC} ICC ) measures rank-order consistency ( ≥ 0.80 – 0.90 \ge 0.80\text{–}0.90 ≥ 0.80 – 0.90 ), while Absolute reliability measures raw trial-to-trial noise via Typical Error ( TE = SD diff / 2 \text{TE} = \text{SD}_{\text{diff}} / \sqrt{2} TE = SD diff / 2 ) and Coefficient of Variation ( CV % = [ TE / Mean ] × 100 % ≤ 5 % \text{CV}\% = [\text{TE} / \text{Mean}] \times 100\% \le 5\% CV % = [ TE / Mean ] × 100% ≤ 5% ) . Comparing Noise ( TE \text{TE} TE ) against the Signal ( SWC = 0.2 × SD between \text{SWC} = 0.2 \times \text{SD}_{\text{between}} SWC = 0.2 × SD between ) determines whether an athlete's observed change exceeds random error ( MDC 95 = 1.96 × 2 × TE ≈ 2.77 × TE \text{MDC}_{95} = 1.96 \times \sqrt{2} \times \text{TE} \approx 2.77 \times \text{TE} MDC 95 = 1.96 × 2 × TE ≈ 2.77 × TE , or ≥ 1.5 – 2.0 × TE \ge 1.5\text{–}2.0 \times \text{TE} ≥ 1.5 – 2.0 × TE in applied monitoring).
Before selecting a new wearable sensor, force plate metric, or field fitness test, the sport scientist must establish confidence in the measurement by evaluating three hierarchical psychometric properties: Validity (accuracy relative to truth), Reliability (precision and repeatability across trials), and Sensitivity / Responsiveness (the signal-to-noise capacity to detect meaningful biological change in an individual athlete).
Taxonomy of Measurement Validity in Sport Science
Validity defines the extent to which an instrument or test protocol accurately measures the specific physiological, biomechanical, or perceptual construct it is intended to assess.
1. Logical (Face) and Content Validity
Face (Logical) Validity: The superficial perception by athletes and coaches that a test appears relevant to their sport. While scientifically subjective, high face validity is essential for securing maximal athlete effort and coaching buy-in.
Content Validity: The systematic evaluation by domain experts that an assessment battery comprehensively samples all critical physiological and mechanical sub-components of competitive performance (for example, verifying that an ice hockey physiological profile includes repeated-sprint skating ability, anaerobic glycolytic power, and lower-body force production rather than solely an upper-body bench press).
2. Criterion Validity (Concurrent and Predictive)
Criterion validity evaluates a practical or novel testing device against an established gold-standard criterion reference .
Concurrent Validity: Determined when the practical field device and the gold-standard criterion are measured simultaneously (e.g., validating a linear position transducer or optical jump mat against a 1000 Hz 1000\text{ Hz} 1000 Hz dual force-plate system, or validating a portable metabolic analyzer against a stationary Douglas bag / gold-standard laboratory cart). Rigorous concurrent validation requires three distinct statistical evaluations:
Linear Correlation ( r r r ): Pearson product-moment correlation ( r r r ) quantifies whether the two devices rank subjects in a similar linear order. Crucial Limitation: A near-perfect correlation ( r = 0.98 r = 0.98 r = 0.98 ) cannot detect systematic offset bias ! If a contact mat overestimates every athlete's jump height by exactly + 6.0 cm +6.0\text{ cm} + 6.0 cm compared to a force plate, r r r will equal 1.00 1.00 1.00 despite severe inaccuracy.
Standard Error of the Estimate ( SEE \text{SEE} SEE / TEE \text{TEE} TEE ): Also termed the Typical Error of the Estimate ( TEE \text{TEE} TEE ) , this metric quantifies the typical residual scatter (prediction error) around the linear regression calibration line in the raw units of the criterion test: SEE = SD y × 1 − r 2 \text{SEE} = \text{SD}_{y} \times \sqrt{1 - r^2} SEE = SD y × 1 − r 2
Bland-Altman Agreement Analysis: The gold-standard graphical and statistical method for evaluating agreement between two measurement tools. For every subject i i i , the difference between the practical device and the criterion ( d i = X practical , i − X criterion , i d_{i} = X_{\text{practical},i} - X_{\text{criterion},i} d i = X practical , i − X criterion , i ) is plotted on the vertical axis against the mean of the two devices on the horizontal axis. Bland-Altman analysis yields two essential parameters:
Systematic (Fixed) Bias ( d ˉ \bar{d} d ˉ ): The mean difference across all subjects, showing whether the practical tool systematically overestimates ( + d ˉ +\bar{d} + d ˉ ) or underestimates ( − d ˉ -\bar{d} − d ˉ ) the gold standard.
95 % 95\% 95% Limits of Agreement ( LoA 95 \text{LoA}_{95} LoA 95 ): The interval within which 95 % 95\% 95% of individual discrepancies between the two devices are expected to lie: LoA 95 = d ˉ ± 1.96 × SD d \text{LoA}_{95} = \bar{d} \pm 1.96 \times \text{SD}_{d} LoA 95 = d ˉ ± 1.96 × SD d Bland-Altman plots also expose **proportional bias (heteroscedasticity)**—where the magnitude of error expands at higher movement velocities or force outputs.
Predictive Validity: Evaluates how well a baseline assessment score predicts a future criterion outcome (such as in-season match high-speed running, draft selection, or prospective non-contact soft-tissue injury). Predictive validity is quantified via regression r 2 r^2 r 2 , odds/hazard ratios, Receiver Operating Characteristic ( ROC \text{ROC} ROC ) Area Under the Curve ( AUC \text{AUC} AUC ) (where AUC = 0.50 \text{AUC} = 0.50 AUC = 0.50 represents random chance and ≥ 0.80 \ge 0.80 ≥ 0.80 indicates strong discrimination), sensitivity (true positive rate), and specificity (true negative rate).
3. Construct Validity (Convergent and Discriminant / Known-Groups)
When evaluating an abstract theoretical quality that lacks a single physical gold standard (such as "sport-specific agility" or "central neuromuscular fatigue"), practitioners establish construct validity :
Convergent Validity: Confirms that the new assessment correlates moderately to strongly with other established tests measuring the same underlying construct.
Discriminant (Known-Groups) Validity: Confirms two properties: (1) the test statistically differentiates between distinct athlete tiers known to differ in that construct ( known-groups validity , such as national-team starters outperforming sub-elite academy players on a reactive agility test); and (2) the test does not simply duplicate an unrelated physical quality ( r 2 < 0.50 r^2 < 0.50 r 2 < 0.50 between linear 20 m 20\text{ m} 20 m sprint time and change-of-direction deficit, proving they measure distinct neuromuscular qualities).
4. Ecological Validity
Ecological validity reflects how faithfully the testing environment, equipment constraints, and perceptual-cognitive stimuli mirror actual competitive sport demands (e.g., skating on ice in full hockey gear while reacting to a visual stimulus versus pedaling a stationary cycle ergometer in a laboratory).
Quantifying Relative vs. Absolute Reliability
Reliability quantifies the consistency or reproducibility of a test across repeated trials when no true biological adaptation or fatigue has occurred. Total observed variance in a dataset equals true between-athlete variance ( σ between 2 \sigma_{\text{between}}^2 σ between 2 ) plus within-athlete error variance ( σ within 2 \sigma_{\text{within}}^2 σ within 2 , which combines equipment technical error and within-subject biological day-to-day fluctuation ).
1. Relative Reliability — Intraclass Correlation Coefficient ( ICC \text{ICC} ICC )
Relative reliability assesses the degree to which individuals maintain their rank order (relative position) within a group across repeated trials, quantified by the Intraclass Correlation Coefficient ( ICC \text{ICC} ICC ) : ICC ≈ σ between 2 σ between 2 + σ within 2 = 1 − TE 2 SD between 2 \text{ICC} \approx \frac{\sigma_{\text{between}}^2}{\sigma_{\text{between}}^2 + \sigma_{\text{within}}^2} = 1 - \frac{\text{TE}^2}{\text{SD}_{\text{between}}^2} ICC ≈ σ between 2 + σ within 2 σ between 2 = 1 − SD between 2 TE 2
Koo & Li (2016) Classification Benchmarks: Based on the 95 % 95\% 95% confidence interval of the ICC \text{ICC} ICC estimate, values < 0.50 < 0.50 < 0.50 indicate Poor , 0.50 – 0.75 0.50\text{–}0.75 0.50 – 0.75 Moderate , 0.75 – 0.90 0.75\text{–}0.90 0.75 – 0.90 Good , and > 0.90 > 0.90 > 0.90 Excellent relative reliability.
Crucial Examination Concept — Sensitivity to Sample Heterogeneity: Because ICC \text{ICC} ICC is a mathematical ratio of between-athlete variance ( SD between 2 \text{SD}_{\text{between}}^2 SD between 2 ) to total variance, ICC \text{ICC} ICC is heavily influenced by the heterogeneity (spread) of the tested group . If a test is administered to a wide, heterogeneous sample (combining recreational athletes, collegiate players, and Olympians), SD between \text{SD}_{\text{between}} SD between is very large, driving the ICC \text{ICC} ICC close to 0.99 0.99 0.99 . Conversely, if the exact same instrument is used on a highly homogeneous elite squad (where all starters have nearly identical 10 m 10\text{ m} 10 m sprint times, yielding a tiny SD between \text{SD}_{\text{between}} SD between ), the ICC \text{ICC} ICC will appear artificially low (e.g., 0.60 0.60 0.60 ) even when absolute trial-to-trial error ( TE \text{TE} TE ) is exceptionally small! Consequently, sport scientists must always pair ICC \text{ICC} ICC with absolute reliability metrics.
2. Absolute Reliability — Typical Error ( TE \text{TE} TE / SEM \text{SEM} SEM ) and Coefficient of Variation ( CV % \text{CV}\% CV % )
Absolute reliability quantifies within-athlete trial-to-trial measurement noise directly in the raw physical units of the test (or as a percentage), completely independent of group heterogeneity.
Typical Error of Measurement ( TE \text{TE} TE ): Popularized in sport science by Will Hopkins, TE \text{TE} TE is calculated from the standard deviation of the within-subject change scores ( d i = Trial 2 , i − Trial 1 , i d_{i} = \text{Trial}_{2,i} - \text{Trial}_{1,i} d i = Trial 2 , i − Trial 1 , i ) across a test-retest reliability study: TE = SD diff 2 \text{TE} = \frac{\text{SD}_{\text{diff}}}{\sqrt{2}} TE = 2 SD diff
Why divide by 2 \sqrt{2} 2 ? Because every difference score ( d i = Trial 2 − Trial 1 d_{i} = \text{Trial}_{2} - \text{Trial}_{1} d i = Trial 2 − Trial 1 ) combines the independent random error variance of two separate trials ( σ diff 2 = σ trial 1 2 + σ trial 2 2 = 2 × TE 2 \sigma_{\text{diff}}^2 = \sigma_{\text{trial } 1}^2 + \sigma_{\text{trial } 2}^2 = 2 \times \text{TE}^2 σ diff 2 = σ trial 1 2 + σ trial 2 2 = 2 × TE 2 ). Taking the square root gives SD diff = 2 × TE \text{SD}_{\text{diff}} = \sqrt{2} \times \text{TE} SD diff = 2 × TE , so the single-trial noise is TE = SD diff / 2 \text{TE} = \text{SD}_{\text{diff}} / \sqrt{2} TE = SD diff / 2 . In classical psychometric theory, TE \text{TE} TE is equivalent to the Standard Error of Measurement ( SEM \text{SEM} SEM ) : SEM = SD pooled × 1 − ICC \text{SEM} = \text{SD}_{\text{pooled}} \times \sqrt{1 - \text{ICC}} SEM = SD pooled × 1 − ICC
Coefficient of Variation ( CV % \text{CV}\% CV % ): Expresses the Typical Error as a dimensionless percentage of the sample grand mean ( X ˉ \bar{X} X ˉ ), allowing practitioners to compare relative noise across tests with different units (e.g., comparing force in Newtons against sprint time in seconds): CV % = ( TE X ˉ ) × 100 % \text{CV}\% = \left(\frac{\text{TE}}{\bar{X}}\right) \times 100\% CV % = ( X ˉ TE ) × 100%
Standard Sport Science Reliability Benchmark: A variable is generally considered highly reliable for applied athlete monitoring when CV % ≤ 5.0 % \text{CV}\% \le 5.0\% CV % ≤ 5.0% (and ICC ≥ 0.80 – 0.90 \text{ICC} \ge 0.80\text{–}0.90 ICC ≥ 0.80 – 0.90 ). If a metric exhibits higher noise (e.g., CV % = 7 % \text{CV}\% = 7\% CV % = 7% for early-phase RFD \text{RFD} RFD ), averaging the mean of k k k trials within a session reduces the effective Typical Error by a factor of k \sqrt{k} k ( TE mean of k = TE / k \text{TE}_{\text{mean of } k} = \text{TE} / \sqrt{k} TE mean of k = TE / k ).
Establishing Confidence in Individual Change (Signal vs. Noise)
Once a test's Noise ( TE \text{TE} TE ) is known, the sport scientist must define the Signal —the minimum change in performance that actually matters in competition—and establish decision thresholds for individual athlete monitoring.
1. Smallest Worthwhile Change ( SWC \text{SWC} SWC )
Based on Cohen's smallest worthwhile effect size ( d = 0.20 d = 0.20 d = 0.20 ) for team-sport athletes, the Smallest Worthwhile Change ( SWC \text{SWC} SWC ) is calculated as one-fifth of the between-athlete standard deviation ( SD between \text{SD}_{\text{between}} SD between ) of a homogeneous peer reference group: SWC 0.2 = 0.2 × SD between \text{SWC}_{0.2} = 0.2 \times \text{SD}_{\text{between}} SWC 0.2 = 0.2 × SD between
(When a more conservative moderate threshold is desired, practitioners use SWC 0.5 = 0.5 × SD between \text{SWC}_{0.5} = 0.5 \times \text{SD}_{\text{between}} SWC 0.5 = 0.5 × SD between ; for solo individual events such as track sprinting or swimming, SWC \text{SWC} SWC is calculated as 0.3 × CV within-athlete race-to-race 0.3 \times \text{CV}_{\text{within-athlete race-to-race}} 0.3 × CV within-athlete race-to-race ).
2. Evaluating Test Sensitivity ( TE \text{TE} TE vs. SWC \text{SWC} SWC )
Comparing the instrument's Noise ( TE \text{TE} TE ) against the practical Signal ( SWC \text{SWC} SWC ) classifies the sensitivity of the assessment:
Good Sensitivity ( TE < SWC \text{TE} < \text{SWC} TE < SWC ): Measurement noise is smaller than a small worthwhile change; the test can confidently detect meaningful adaptations from a single measurement.
Marginal / OK Sensitivity ( TE ≈ SWC \text{TE} \approx \text{SWC} TE ≈ SWC ): Noise equals the signal; the practitioner should average 2 to 4 2\text{ to } 4 2 to 4 repeated trials per session so that TE / k < SWC \text{TE} / \sqrt{k} < \text{SWC} TE / k < SWC .
Poor Sensitivity ( TE > SWC \text{TE} > \text{SWC} TE > SWC ): Measurement noise masks small worthwhile changes; only moderate-to-large changes can be detected.
3. Minimal Detectable Change ( MDC 95 \text{MDC}_{95} MDC 95 ) vs. Applied Monitoring Thresholds
Minimal Detectable Change at 95 % 95\% 95% Confidence ( MDC 95 \text{MDC}_{95} MDC 95 / Smallest Real Difference): Defines the statistical threshold above which an observed difference between two single measurements ( Δ = X 2 − X 1 \Delta = X_{2} - X_{1} Δ = X 2 − X 1 ) has less than a 5 % 5\% 5% probability of occurring due to random measurement error alone: MDC 95 = 1.96 × 2 × TE ≈ 2.77 × TE \text{MDC}_{95} = 1.96 \times \sqrt{2} \times \text{TE} \approx 2.77 \times \text{TE} MDC 95 = 1.96 × 2 × TE ≈ 2.77 × TE
At the 90 % 90\% 90% confidence level, the threshold is: MDC 90 = 1.645 × 2 × TE ≈ 2.33 × TE \text{MDC}_{90} = 1.645 \times \sqrt{2} \times \text{TE} \approx 2.33 \times \text{TE} MDC 90 = 1.645 × 2 × TE ≈ 2.33 × TE
Applied Daily Monitoring Bands (Hopkins Framework): In high-performance sport, waiting for an athlete to change by 2.77 × TE 2.77 \times \text{TE} 2.77 × TE is often overly conservative and risks overlooking early neuromuscular fatigue (Type II error). In applied monitoring dashboards, sport scientists plot the athlete's score with an error bar of ± 1.0 × TE \pm 1.0 \times \text{TE} ± 1.0 × TE (or ± 1.5 to 2.0 × TE \pm 1.5\text{ to } 2.0 \times \text{TE} ± 1.5 to 2.0 × TE , roughly 87 – 95 % 87\text{–}95\% 87 – 95% confidence for a single score) against the shaded ± SWC \pm \text{SWC} ± SWC band. Whenever an athlete's change ∣ Δ ∣ |\Delta| ∣Δ∣ exceeds both the SWC \text{SWC} SWC and the chosen TE \text{TE} TE error threshold , the practitioner reports a clear, actionable real change .
Feasibility and Selecting KPIs from Validated Tools
Task 3D ends with two practical judgments: whether a tool is feasible in your setting, and which of its outputs should become key performance indicators (KPIs) . A tool can be valid and reliable in a laboratory and still fail in your program.
Feasibility Question What to Check Warning Sign
Time and throughput Setup time, minutes per athlete, number of trials, and whether testing fits inside a warm-up A 3-minute protocol per athlete for a 60-player squad in a 45-minute block
Athlete burden Fatigue cost, invasiveness, discomfort, and compliance Daily blood draws, or maximal tests during congested fixtures
Staff skill and processing Who runs, cleans, and reports the data, and how quickly Data that cannot be processed before the next decision
Cost and portability Purchase price, licenses, consumables, and travel use A fixed laboratory system for a team that trains on the road
Data access Raw export, API access, and firmware control Only a vendor composite score is available
When choosing KPIs from a feasible tool, keep a metric only if it passes four checks:
Relevance: It maps onto a determinant identified in the needs analysis, such as eccentric braking or maximal sprint speed.
Measurement quality: It has acceptable validity and absolute reliability, with TE ≤ SWC \text{TE} \le \text{SWC} TE ≤ SWC or a manageable TE / k \text{TE}/\sqrt{k} TE / k after averaging trials.
Sensitivity: It changes with training or fatigue in your population.
Actionability: A pre-agreed threshold triggers a specific decision.
For example, a force-plate CMJ export may offer more than 100 variables. A feasible KPI set might be jump height (outcome), RSI mod \text{RSI}_{\text{mod}} RSI mod (strategy), and braking impulse or eccentric RFD (braking). Each should pass the reliability and SWC checks above, and none should be a near-duplicate of another.
Worked Calculation Example: Countermovement Jump Psychometrics
Scenario: A sport scientist conducts a test-retest reliability trial of countermovement jump ( CMJ \text{CMJ} CMJ ) height ( 48 hours 48\text{ hours} 48 hours apart) on N = 20 N = 20 N = 20 varsity basketball players:
Group Grand Mean ( X ˉ \bar{X} X ˉ ) = 50.0 cm = 50.0\text{ cm} = 50.0 cm
Between-Athlete Standard Deviation ( SD between \text{SD}_{\text{between}} SD between ) = 6.0 cm = 6.0\text{ cm} = 6.0 cm
Standard Deviation of Trial 2 minus Trial 1 Difference Scores ( SD diff \text{SD}_{\text{diff}} SD diff ) = 1.414 cm = 1.414\text{ cm} = 1.414 cm
Step 1 — Calculate Typical Error ( TE \text{TE} TE ): TE = SD diff 2 = 1.414 cm 1.4142 ≈ 1.00 cm \text{TE} = \frac{\text{SD}_{\text{diff}}}{\sqrt{2}} = \frac{1.414\text{ cm}}{1.4142} \approx 1.00\text{ cm} TE = 2 SD diff = 1.4142 1.414 cm ≈ 1.00 cm
Step 2 — Calculate Coefficient of Variation ( CV % \text{CV}\% CV % ): CV % = ( 1.00 cm 50.0 cm ) × 100 % = 2.0 % ( Passes ≤ 5.0 % benchmark! ) \text{CV}\% = \left(\frac{1.00\text{ cm}}{50.0\text{ cm}}\right) \times 100\% = 2.0\% \quad (\text{Passes } \le 5.0\% \text{ benchmark!}) CV % = ( 50.0 cm 1.00 cm ) × 100% = 2.0% ( Passes ≤ 5.0% benchmark! )
Step 3 — Calculate Intraclass Correlation Coefficient ( ICC \text{ICC} ICC ): ICC ≈ 1 − TE 2 SD between 2 = 1 − 1.00 2 6.0 2 = 1 − 1.00 36.0 ≈ 0.97 ( Excellent relative reliability! ) \text{ICC} \approx 1 - \frac{\text{TE}^2}{\text{SD}_{\text{between}}^2} = 1 - \frac{1.00^2}{6.0^2} = 1 - \frac{1.00}{36.0} \approx 0.97 \quad (\text{Excellent relative reliability!}) ICC ≈ 1 − SD between 2 TE 2 = 1 − 6. 0 2 1.0 0 2 = 1 − 36.0 1.00 ≈ 0.97 ( Excellent relative reliability! )
Step 4 — Calculate Smallest Worthwhile Change ( SWC 0.2 \text{SWC}_{0.2} SWC 0.2 ) and Rate Sensitivity: SWC 0.2 = 0.2 × 6.0 cm = 1.20 cm \text{SWC}_{0.2} = 0.2 \times 6.0\text{ cm} = 1.20\text{ cm} SWC 0.2 = 0.2 × 6.0 cm = 1.20 cm
Because TE ( 1.00 cm ) < SWC ( 1.20 cm ) \text{TE} (1.00\text{ cm}) < \text{SWC} (1.20\text{ cm}) TE ( 1.00 cm ) < SWC ( 1.20 cm ) , the test exhibits Good sensitivity to detect small meaningful changes!
Step 5 — Calculate Minimal Detectable Change ( MDC 95 \text{MDC}_{95} MDC 95 and MDC 90 \text{MDC}_{90} MDC 90 ): MDC 95 = 1.96 × 2 × 1.00 cm ≈ 2.77 cm and MDC 90 = 1.645 × 2 × 1.00 cm ≈ 2.33 cm \text{MDC}_{95} = 1.96 \times \sqrt{2} \times 1.00\text{ cm} \approx 2.77\text{ cm} \quad \text{and} \quad \text{MDC}_{90} = 1.645 \times \sqrt{2} \times 1.00\text{ cm} \approx 2.33\text{ cm} MDC 95 = 1.96 × 2 × 1.00 cm ≈ 2.77 cm and MDC 90 = 1.645 × 2 × 1.00 cm ≈ 2.33 cm
Step 6 — Interpret Individual Athlete Pre- to Post-Mesocycle Changes:
Athlete A improves from 48.0 cm 48.0\text{ cm} 48.0 cm to 48.8 cm 48.8\text{ cm} 48.8 cm ( Δ = + 0.80 cm \Delta = +0.80\text{ cm} Δ = + 0.80 cm ): Because + 0.80 cm +0.80\text{ cm} + 0.80 cm is less than both TE \text{TE} TE ( 1.00 cm 1.00\text{ cm} 1.00 cm ) and SWC \text{SWC} SWC ( 1.20 cm 1.20\text{ cm} 1.20 cm ), the change is unclear / within random measurement noise .
Athlete B improves from 48.0 cm 48.0\text{ cm} 48.0 cm to 51.0 cm 51.0\text{ cm} 51.0 cm ( Δ = + 3.00 cm \Delta = +3.00\text{ cm} Δ = + 3.00 cm ): Because + 3.00 cm +3.00\text{ cm} + 3.00 cm exceeds both SWC \text{SWC} SWC ( 1.20 cm 1.20\text{ cm} 1.20 cm ) and MDC 95 \text{MDC}_{95} MDC 95 ( 2.77 cm 2.77\text{ cm} 2.77 cm ), the practitioner has > 95 % > 95\% > 95% statistical confidence that a practically meaningful real adaptation occurred .
Summary of Psychometric Formulas and Benchmarks
Psychometric Metric Mathematical Formula Target Benchmark / Interpretation Rule
Bland-Altman 95 % 95\% 95% LoA \text{LoA} LoA d ˉ ± 1.96 × SD d \bar{d} \pm 1.96 \times \text{SD}_{d} d ˉ ± 1.96 × SD d Evaluates concurrent validity systematic bias ( d ˉ \bar{d} d ˉ ) and 95 % 95\% 95% random error bounds against a gold standard
Standard Error of Estimate ( SEE \text{SEE} SEE ) SD y × 1 − r 2 \text{SD}_{y} \times \sqrt{1 - r^2} SD y × 1 − r 2 Quantifies typical calibration prediction error in raw units of the criterion reference
Intraclass Correlation ( ICC \text{ICC} ICC ) 1 − ( TE 2 / SD between 2 ) 1 - (\text{TE}^2 / \text{SD}_{\text{between}}^2) 1 − ( TE 2 / SD between 2 ) Relative rank-order reliability: < 0.50 < 0.50 < 0.50 Poor, 0.50 – 0.75 0.50\text{–}0.75 0.50 – 0.75 Moderate, 0.75 – 0.90 0.75\text{–}0.90 0.75 – 0.90 Good, > 0.90 > 0.90 > 0.90 Excellent
Typical Error ( TE \text{TE} TE / SEM \text{SEM} SEM ) SD diff / 2 = SD × 1 − ICC \text{SD}_{\text{diff}} / \sqrt{2} = \text{SD} \times \sqrt{1 - \text{ICC}} SD diff / 2 = SD × 1 − ICC Absolute within-subject trial-to-trial noise in raw physical measurement units
Coefficient of Variation ( CV % \text{CV}\% CV % ) ( TE / Grand Mean ) × 100 % (\text{TE} / \text{Grand Mean}) \times 100\% ( TE / Grand Mean ) × 100% Dimensionless percentage noise; standard sport science target is CV % ≤ 5.0 % \text{CV}\% \le 5.0\% CV % ≤ 5.0%
Smallest Worthwhile Change ( SWC \text{SWC} SWC ) 0.2 × SD between 0.2 \times \text{SD}_{\text{between}} 0.2 × SD between Minimum practically meaningful performance signal ( 0.5 × SD between 0.5 \times \text{SD}_{\text{between}} 0.5 × SD between for moderate threshold)
Minimal Detectable Change ( MDC 95 \text{MDC}_{95} MDC 95 ) 1.96 × 2 × TE ≈ 2.77 × TE 1.96 \times \sqrt{2} \times \text{TE} \approx 2.77 \times \text{TE} 1.96 × 2 × TE ≈ 2.77 × TE Threshold for 95 % 95\% 95% statistical certainty that an individual change exceeds two-trial random noise