Study guide
5.2 Content, Criterion-Related (Predictive/Concurrent), and Construct Validity in Public Selection
PSHRA Certified Professional PSHRA-CP
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Introduction to Public Sector HR & Merit Systems
1.1 PSHRA-CP Exam Structure, Public Sector HR Foundations & Merit Principles 1.2 Historical Evolution: Spoils System, Pendleton Act & Civil Service Reform Act of 1978 1.3 The 9 Statutory Merit System Principles & Civil Service Administration 1.4 Prohibited Personnel Practices (5 U.S.C. § 2302) & Whistleblower Protection
2 Chapter 2: Job Analysis & Minimum Qualifications
2.1 Public Sector Job Analysis Methods: Task Inventories, Functional Job Analysis & O*NET 2.2 KSAO Development, Essential Functions & Duty-to-KSAO Linkage Matrices 2.3 Establishing Job-Related Minimum Qualifications & Screening Thresholds 2.4 Crafting Public Sector Job Descriptions, Class Specifications & FLSA/ADA Compliance
3 Chapter 3: Recruitment Strategy, Sourcing & Diversity
3.1 Public Sector Sourcing & Outreach to Underrepresented Groups 3.2 Job Announcements, Examination Bulletins & EEO Compliance 3.3 Open-Competitive vs. Promotional Hiring Paths 3.4 Pathways Programs, Internships & Apprenticeships in Government
4 Chapter 4: Competitive Testing & Selection Instruments
4.1 Written & Computer-Based Job Knowledge and Cognitive Ability Testing in Civil Service 4.2 Work Sample Tests, Performance Simulations, Assessment Centers & Physical Ability Tests 4.3 Structured Interviewing, Behavior Description (BDI), Situational (SI) & Panel Standardization 4.4 Training & Experience (T&E) Evaluations, Point-Factor Scoring & Supplemental Questionnaires
5 Chapter 5: Test Validation, Psychometrics & Adverse Impact
5.1 Uniform Guidelines on Employee Selection Procedures (UGESP 1978) & Validation Standards 5.2 Content, Criterion-Related (Predictive/Concurrent), and Construct Validity in Public Selection 5.3 Test Reliability Metrics, Standard Error of Measurement, Cut Scores & Banding 5.4 Four-Fifths (80%) Rule, Disparate Impact Analysis, Standard Deviation Rule & Adverse Impact Remediation
6 Chapter 6: Candidate Ranking, Certification & Selection
6.1 Civil Service Certification: Traditional Rule-of-Three vs. Modern Category Rating Systems 6.2 Veterans' Preference Points, Preference Categories & Pass-Over Procedures 6.3 Eligible Lists, Order of Precedence, Canvassing & Selective Certification 6.4 The Probationary Period, Rejection During Probation & Due Process Protections
7 Chapter 7: Public Sector Employment Law & Compliance
7.1 Title VII of the Civil Rights Act of 1964, EEOC Procedures, Pattern or Practice & Public Agency EEO Compliance 7.2 ADA/ADAAA in Public Sector: Essential Functions, Reasonable Accommodation Interactive Dialogue & Undue Hardship Standards 7.3 FMLA, FLSA Overtime/Comp Time Rules (29 U.S.C. § 207(o)), Pregnant Workers Fairness Act & PUMP Act 7.4 Age Discrimination in Employment Act (ADEA), GINA, USERRA Military Leave Protections & 42 U.S.C. § 1983 Actions
8 Chapter 8: Constitutional Law, Public Ethics & Documentation
8.1 Fifth and Fourteenth Amendment Due Process: Property/Liberty Interests & Pre-Disciplinary Loudermill Rights 8.2 First Amendment Speech & Association: Pickering-Connick Public Concern Test, Garcetti Official Duty Rule & Political Patronage Bans 8.3 Fourth Amendment Privacy: O'Connor v. Ortega Workplace Searches, Special Needs Drug Testing & Electronic Monitoring 8.4 The Hatch Act (5 U.S.C. §§ 7321-7326), State 'Little Hatch Acts', Public Records/FOIA & Conflict of Interest Codes 8.5 Personnel Policy Development, Administrative Rulemaking & HR Documentation Standards
9 Chapter 9: Public Sector Labor Relations & Collective Bargaining
9.1 Public Sector Labor Frameworks: CSRA Title VII, Federal Labor Relations Authority & State Public Employment Relations Boards (PERBs) 9.2 Bargaining Unit Determination, Community of Interest, Certification Elections & Duty of Fair Representation 9.3 Scope of Bargaining: Mandatory, Permissive, Prohibited Subjects, Management Rights & Impact/Implementation Bargaining 9.4 Impasse Resolution: Mediation, Fact-Finding, Interest Arbitration, Right-to-Strike Bans & Post-Janus v. AFSCME Agency Fee Landscape
10 Chapter 10: Discipline, Grievances & Performance Management
10.1 The Just Cause Standard: Daugherty's Seven Tests of Just Cause in Public Agency Discipline 10.2 Progressive Discipline Hierarchy, Written Reprimands to Discharges, Loudermill Notice and Hearing Execution 10.3 Contractual and Civil Service Grievance Procedures, Step Escalation, MSPB Appeals & Binding Rights Arbitration 10.4 Public Sector Performance Appraisal Systems, Standards Alignment, Performance Improvement Plans (PIPs) & Unacceptable Performance Removals (5 U.S.C. Ch. 43 vs Ch. 75)
11 Chapter 11: Job Classification & Compensation Systems
11.1 Public Sector Job Evaluation: Point-Factor Systems, Factor Evaluation System (FES) & Position Description Audits 11.2 General Schedule (GS) Grade-and-Step Mechanics, Locality Pay, Special Rates, Broadbanding & Career Ladders 11.3 Conducting Market Salary Surveys, Benchmark Positioning, Compa-Ratios & Balancing External Competitiveness with Internal Equity 11.4 FLSA Exemption Audits (Executive, Administrative, Professional), Public Comp Time Accrual/Payouts & 7(k) Public Safety Rules 11.5 Equal Pay Act in Public Agencies, Pay Transparency Statutes, Salary History Bans & Structural Compression Resolution
12 Chapter 12: Employee Benefits & Total Rewards in Government
12.1 Public Pension Systems: FERS vs. CSRS, State/Local Defined Benefit Formulas, Vesting & Post-Retirement COLAs 12.2 Government Deferred Compensation: Section 457(b) Eligible Plans, 401(a) Defined Contribution & Catch-Up Provisions 12.3 Federal Employees Health Benefits (FEHB) & Public Agency Group Health, FSAs, HSAs & Employee Wellness Programs 12.4 Other Post-Employment Benefits (OPEB), GASB 74/75 Reporting Liabilities, Public Disability & Life Insurance Systems
13 Chapter 13: HRIS, Workforce Analytics & Public Administration
13.1 Public Sector HRIS Architecture, Enterprise ERP Integration, Civil Service ATS Platforms & Compliant Onboarding 13.2 Core Public HR Metrics: Time-to-Hire, Turnover Rates, Cost-per-Hire, Diversity Indices & Civil Service Reporting 13.3 Government Workforce & Succession Planning: Demographic Cliff, Retirement Forecasting, Knowledge Transfer & Leadership Continuity 13.4 Public HR Data Security, PII Protection, Records Retention Schedules, FOIA/Public Records Compliance & Audit Readiness
5.2 Content, Criterion-Related (Predictive/Concurrent), and Construct Validity in Public Selection
Key Takeaways Explain This Topic AI
Validity is the degree to which accumulated empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and decisions made based on test scores.
Content validity demonstrates that test items represent a direct, representative sample of critical job behaviors, duties, or knowledge/skills identified through a comprehensive job analysis, and is quantified via Lawshe's Content Validity Ratio (CVR).
Criterion-related validity establishes a statistical correlation (validity coefficient r) between test scores (predictor) and actual job performance metrics (criterion), utilizing either predictive (prospective) or concurrent (incumbent) research designs.
Construct validity proves that an assessment instrument measures an underlying psychological construct (e.g., cognitive ability, spatial reasoning, emotional intelligence) and that this construct is critical for successful job performance.
UGESP explicitly prohibits relying on content validity for tests measuring abstract mental traits, psychological constructs, or knowledge/skills that can be easily acquired during brief post-hire orientation.
Last updated: August 2026
5.2 Content, Criterion-Related (Predictive/Concurrent), and Construct Validity in Public Selection
In public personnel management, test validation is the scientific and legal process of accumulating empirical evidence to demonstrate that inferences, rankings, and employment decisions drawn from assessment scores are justified, accurate, and job-related. Under Section 14 of the Uniform Guidelines on Employee Selection Procedures (UGESP), public employers defending a selection procedure that produces adverse impact must establish validity through one of three foundational validation strategies:
- Content Validity
- Criterion-Related Validity (Predictive or Concurrent)
- Construct Validity
Each validation model possesses distinct methodological requirements, statistical benchmarks, practical advantages, and legal limitations within civil service environments.
1. The Tripartite Validation Framework +-----------------------------------------------------------------------------------------+
- | THE THREE CLASSICAL VALIDATION STRATEGIES (UGESP) |
- | |
- | +------------------------------------+ +------------------------------------+ |
- | | CONTENT VALIDITY | | CRITERION-RELATED VALIDITY | |
- | | - Direct behavioral sample of job | | - Statistical correlation between | |
- | | - Based on thorough job analysis | | test score (X) & performance (Y) | |
- | | - Work samples, knowledge tests | | - Predictive or Concurrent designs | |
- | | - Quantified via Lawshe's CVR | | - Evaluated by correlation coeff r | |
- | +------------------------------------+ +------------------------------------+ |
- | \ / |
- | \ / |
- | v v |
- | +-----------------------------------------+ |
- | | CONSTRUCT VALIDITY | |
- | | - Measures underlying psychological | |
- | | trait/construct (GMA, integrity) | |
- | | - Requires convergent/discriminant proof| |
- | | - Most complex and demanding validation | |
- | +-----------------------------------------+ |
- +-----------------------------------------------------------------------------------------+
- 2. Content Validity: Principles and Public Sector Application
Content validity demonstrates that the content of the selection instrument is a representative sample of the critical work behaviors, operational duties, or prerequisite Knowledge, Skills, and Abilities (KSAs) required for effective performance in the target position.
Core Prerequisites for Content Validity under UGESP Section 14C:
Rigorous Job Analysis: Must identify critical job tasks, their frequency and importance, and the specific KSAs required to execute each task.
Direct Operational Linkage: The test content must closely replicate the actual work environment, duties, or knowledge domains (e.g., a typing test for an administrative assistant, a drafting simulation for an engineer, a physical agility test for a firefighter).
Pre-Hire Prerequisite Rule: Content validity cannot be used to assess knowledge, skills, or abilities that are routinely taught during post-hire onboarding or learned on the job within a brief orientation period.
The Abstract Trait Prohibition: UGESP explicitly states that content validity is inappropriate for tests measuring abstract mental traits, psychological constructs, or general aptitudes (e.g., intelligence, judgment, leadership orientation, neuroticism). +-----------------------------------------------------------------------------------------+
- | LAWSHE'S CONTENT VALIDITY RATIO (CVR) METHODOLOGY |
- | |
- | A panel of Subject Matter Experts (SMEs) evaluates each proposed test item using |
- | a 3-point scale: |
- | 1. Essential to job performance |
- | 2. Useful but not essential |
- | 3. Not necessary |
- | |
- | n_e - (N / 2) |
- | CVR = --------------- |
- | N / 2 |
- | |
- | Where: |
- | n_e = Number of SME panelists rating the item as "Essential" |
- | N = Total number of SME panelists on the review board |
- +-----------------------------------------------------------------------------------------+
Interpreting Lawshe's CVR:
CVR Range: Ranges from -1.00 (all SMEs rate item "Not Necessary") to 0.00 (exactly half rate "Essential") to +1.00 (all SMEs rate "Essential").
Statistical Significance Threshold: An item is retained only if its CVR exceeds the minimum statistical threshold based on panel size ( p < .05 p < .05 p < .05 , one-tailed test):
- SME Panel Size ( N N N ) Minimum Required CVR Value ( p < .05 p < .05 p < .05 )
- 5 SMEs 0.99 (Unanimous agreement required)
- 10 SMEs 0.62
- 15 SMEs 0.49
- 20 SMEs 0.42
- 30 SMEs 0.33
- 40+ SMEs 0.29
- Content Validity Index (CVI)
The Content Validity Index (CVI) represents the arithmetic mean of all retained item CVR values across the final assessment instrument, providing an overall index of the test's content representativeness.
3. Criterion-Related Validity: Predictive vs. Concurrent Designs
Criterion-related validity demonstrates a statistically significant empirical relationship between scores on an assessment instrument (the predictor , X X X ) and subsequent measures of actual work performance (the criterion , Y Y Y ). +-----------------------------------------------------------------------------------------+
- | CRITERION-RELATED VALIDITY: RESEARCH DESIGN COMPARISON |
- | |
- | [PREDICTIVE VALIDITY DESIGN (PROSPECTIVE)] |
- | |
- | Step 1: Administer Test (X) to All APPLICANTS at Initial Screening |
- | Step 2: Hire Applicants WITHOUT Using Test Scores (or Hire All Applicants) |
- | Step 3: Allow Employees to Work in Target Classification (6-12 Months) |
- | Step 4: Collect Objective Criterion Performance Data (Y) (Ratings, Errors, Output) |
- | Step 5: Compute Correlation Coefficient r_xy between Applicant Score & Performance |
- | |
- | ----------------------------------------------------------------------------------- |
- | |
- | [CONCURRENT VALIDITY DESIGN (INCUMBENT)] |
- | |
- | Step 1: Select Representative Sample of CURRENT TENURED EMPLOYEES |
- | Step 2: Administer Test (X) to Current Incumbents |
- | Step 3: Collect SIMULTANEOUS Performance Evaluations (Y) for Same Incumbents |
- | Step 4: Compute Correlation Coefficient r_xy between Incumbent Score & Performance |
- +-----------------------------------------------------------------------------------------+
- Comprehensive Comparison Matrix: Predictive vs. Concurrent
Evaluation Dimension Predictive Validity Design Concurrent Validity Design
Test Population Job Applicants (pre-hire). Current Job Incumbents (post-hire).
Timing of Data Predictor collected at baseline; criterion collected months later. Predictor and criterion collected simultaneously.
Range Restriction Minimal. Evaluates full spectrum of applicant abilities. Severe. Poor performers have been fired; top performers promoted.
Test-Taker Motivation High (applicants striving to secure employment). Variable/Low (incumbents may experience test fatigue or anxiety).
Organizational Cost & Time High cost; significant time lag before validation results emerge. Fast turnaround; immediate empirical results.
Legal/Scientific Robustness Considered the gold standard of criterion validation. Legally acceptable under UGESP, but subject to range restriction critiques.
Statistical Evaluation of Validity Coefficients ( r x y r_{xy} r x y )
Criterion validity is quantified using the Pearson product-moment correlation coefficient ( r x y r_{xy} r x y ) between the predictor and criterion: r x y = ∑ ( X − X ˉ ) ( Y − Y ˉ ) ∑ ( X − X ˉ ) 2 ∑ ( Y − Y ˉ ) 2 r_{xy} = \frac{\sum (X - \bar{X})(Y - \bar{Y})}{\sqrt{\sum (X - \bar{X})^2 \sum (Y - \bar{Y})^2}} r x y = ∑ ( X − X ˉ ) 2 ∑ ( Y − Y ˉ ) 2 ∑ ( X − X ˉ ) ( Y − Y ˉ )
Statistical Significance: UGESP requires that the correlation coefficient be statistically significant at p < .05 p < .05 p < .05 (meaning there is less than a 5% probability that the correlation occurred by random sampling error).
Coefficient of Determination ( r 2 r^2 r 2 ): Represents the proportion of variance in job performance explained by the test score (e.g., if r = .50 r = .50 r = .50 , r 2 = .25 r^2 = .25 r 2 = .25 , meaning the test explains 25% of the performance variance).
Professional Benchmarks for Validity Coefficients ( r r r ):
Validity Coefficient ( r r r ) Practical Interpretation in Employee Selection
r < .10 r < .10 r < .10 Negligible / Unusable for selection decisions.
r = .11 − .20 r = .11 - .20 r = .11 − .20 Weak / Minimally useful as a standalone hurdle.
r = .21 − .34 r = .21 - .34 r = .21 − .34 Acceptable / Useful when combined in a composite battery.
r = .35 − .50 r = .35 - .50 r = .35 − .50 Strong / Highly effective predictor of performance.
r > .50 r > .50 r > .50 Exceptional / Rare in real-world selection systems.
Range Restriction (Attenuation) and Correction Formulas
When calculating validity coefficients on current employees (concurrent validity) or pre-screened applicants, the observed correlation is artificially depressed because low-scoring individuals are absent from the sample. Industrial psychologists apply Thorndike's Case II Range Restriction Correction Formula to estimate the true operational validity coefficient ( r U r_U r U ) in the unrestricted applicant pool: r U = r R ( S D U S D R ) 1 + r R 2 ( ( S D U S D R ) 2 − 1 ) r_U = \frac{r_R \left(\frac{SD_U}{SD_R}\right)}{\sqrt{1 + r_R^2 \left(\left(\frac{SD_U}{SD_R}\right)^2 - 1\right)}} r U = 1 + r R 2 ( ( S D R S D U ) 2 − 1 ) r R ( S D R S D U )
Where r R r_R r R is the restricted sample correlation, S D R SD_R S D R is the standard deviation in the restricted sample, and S D U SD_U S D U is the standard deviation in the unrestricted applicant pool.
4. Criterion Measurement: Contamination, Deficiency, and Reliability
A criterion validation study is only as sound as the job performance measure ( criterion ) it utilizes. UGESP Section 14B(3) requires that criteria represent critical aspects of work performance. +-----------------------------------------------------------------------------------------+
- | CRITERION DYNAMICS IN VALIDATION STUDIES |
- | |
- | +-------------------------------------------------------------------------+ |
- | | THE ACTUAL WORK DOMAIN | |
- | | +---------------------------------+ | |
- | | | CRITERION DEFICIENCY | | |
- | | | (Critical job facets omitted) | | |
- | | | - Customer de-escalation skill | | |
- | | | - Team safety adherence | | |
- | | +---------------------------------+ | |
- | | | | |
- | | v | |
- | | +=================================+ <--- [VALID CRITERION OVERLAP]| |
- | | | CRITERION RELEVANCE | (Accurate measure of | |
- | | | (Core job duties captured) | true job competence) | |
- | | +=================================+ | |
- | +-----------------|-------------------------------------------------------+ |
- | | |
- | v |
- | +-----------------------------------------+ |
- | | CRITERION CONTAMINATION | |
- | | (Non-job factors polluting the metric) | |
- | | - Rater personal favoritism | |
- | | - Shift equipment differences | |
- | | - Unequal territory sales opportunities | |
- | +-----------------------------------------+ |
- +-----------------------------------------------------------------------------------------+
Criterion Relevance: The degree to which the criterion measures the essential performance dimensions of the job.
Criterion Deficiency: Failure of the criterion to measure important job duties (e.g., evaluating a police officer exclusively on the number of traffic citations issued, omitting community problem-solving and de-escalation).
Criterion Contamination: Inclusion of extraneous, non-job-related variance in the criterion score (e.g., subjective supervisory ratings influenced by employee personality, race, gender, or differences in mechanical equipment quality across shifts).
5. Construct Validity: Psychological Constructs in Selection
Construct validity demonstrates that an assessment instrument measures an underlying psychological construct, trait, or characteristic (e.g., general mental ability, mechanical comprehension, conscientiousness, emotional intelligence) AND that this construct is critical for successful job performance.
UGESP Section 14D Stringency
Construct validity is the most intellectually rigorous and legally complex validation strategy. To establish construct validity under UGESP, the agency must provide a three-part chain of evidence:
Demonstrate the Construct Exists: Theoretical and empirical proof that the construct represents a distinct, coherent psychological domain.
Demonstrate Test Measures the Construct: Evidence of convergent validity (high correlation with established tests measuring the same construct) and discriminant validity (low correlation with tests measuring unrelated constructs).
Demonstrate Construct Predicts Job Performance: Empirical criterion-related evidence linking the construct to successful performance across the target job family.
6. Validity Generalization (VG) and Synthetic Validation
In modern public HR administration, small agencies frequently lack the large sample sizes required to conduct localized, highly powered criterion-related validation studies ( N ≥ 100 N \ge 100 N ≥ 100 ).
Validity Generalization (Hunter-Schmidt Model)
Meta-analytic research demonstrates that validities for well-defined cognitive and behavioral predictors do not vary randomly from organization to organization (the "situational specificity hypothesis" is largely a statistical artifact of small sample error and range restriction). Under professional SIOP standards, an agency can transport validity evidence from broad meta-analyses if the target job is demonstrably similar in essential work duties and required KSAOs.
UGESP Transportability Standards (Section 7)
To legally transport a validity study from another jurisdiction under UGESP, the receiving public agency must prove:
Job Comparability: Thorough job analyses demonstrate that the essential duties and critical work behaviors of the two positions are substantially identical.
Test Comparability: The identical assessment instrument is utilized under standardized administration conditions.
Applicant Pool Comparability: The demographic and educational characteristics of the applicant pools are substantially similar.