Study guide

18.2 Internal CDI Quality Audits and Reviewer Calibration

AHIMA Certified Documentation Integrity Practitioner CDIP

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Authoritative Coding Resources & Conventions

1.1 Official Coding Resources: Scope, Authority & Research 1.2 General Coding Conventions, Punctuation & Instructional Notes 1.3 Chapter-Specific and Setting-Specific Coding Guidelines

2 Chapter 2: Principal & Secondary Diagnosis Determination

2.1 UHDDS Definitions & Principal Diagnosis Selection 2.2 Two or More Diagnoses Equally Meeting Criteria 2.3 Secondary Diagnosis Reporting Standards & POA Status

3 Chapter 3: Procedure Coding & Sequencing Standards

3.1 ICD-10-PCS Root Operations & Inpatient Procedure Coding 3.2 Principal Procedure Selection & Surgical Sequencing 3.3 Applying Official References to Ambiguous Records

4 Chapter 4: Inpatient Prospective Payment & DRG Grouping

4.1 MS-DRG Logic, CC/MCC Exclusions & Severity Levels 4.2 Working vs. Final DRG Assignment & Optimization vs. Maximization 4.3 CDI-Coding Reconciliation & Annual Regulatory Updates

5 Chapter 5: Enterprise CDI Promotion & Program Value

5.1 Articulating CDI Value to C-Suite & Hospital Leadership 5.2 Organizational Models, Inpatient/Outpatient Scope & Staffing 5.3 Cross-Functional Collaboration: HIM, Quality & Case Management

6 Chapter 6: Cross-Functional Alignment & Improvement Projects

6.1 Partnering with Nursing, Pharmacy & Allied Health Professionals 6.2 Documentation Improvement Project Identification & Scoping 6.3 Executing, Piloting & Measuring CDI Project Outcomes

7 Chapter 7: Physician Champion Engagement & Clinical Advocacy

7.1 Selection, Role & Credibility of the CDI Physician Champion 7.2 Peer-to-Peer Engagement, Coaching & Overcoming Resistance 7.3 Developing Standard CDI Operational Policies & Procedures

8 Chapter 8: Compliant Policy Architecture & Query Governance

8.1 AHIMA/ACDIS Practice Brief Standards in Policy Construction 8.2 Medical Record Integration of Provider Query Responses 8.3 Policies for Electronic, Paper, and Verbal Query Communications

9 Chapter 9: Query Escalation & Professional Communication

9.1 Chain of Command & Protocols for Unanswered Queries 9.2 Conflict Resolution & Professional Feedback Mechanisms 9.3 Assessing Stakeholder Learning Needs & Specialty Curricula

10 Chapter 10: Educational Delivery & Broader Healthcare Impacts

10.1 Delivery Methods: Microlearning, Case Studies & Grand Rounds 10.2 Documentation Impacts on Clinical Research & Epidemiology 10.3 Documentation Impacts on Case Management, LOS & Reimbursement

11 Chapter 11: Clinical Record Synthesis & Encounter Trajectory

11.1 Synthesizing Provider Notes, Consults & Progress Trajectory 11.2 Interpreting Diagnostic Imaging, Pathology & Labs 11.3 Resolving Conflicting, Ambiguous & Copy-Forward Entries

12 Chapter 12: Advanced Clinical Concepts & Review Prioritization

12.1 Pharmacotherapy, Clinical Trajectories & Disease Correlations 12.2 Inpatient Review Prioritization: High-Risk & High-Volume 12.3 Payer Prioritization, Length of Stay & Re-Review Cadence

13 Chapter 13: Clinical Indicator Mapping & Gap Recognition

13.1 Systematic Identification of Documentation Gaps 13.2 Sepsis Definitions, SEP-1, and Coding 13.3 AKI Criteria, Tubular Injury, and CKD Staging

14 Chapter 14: Diagnostic Precision in High-Impact Conditions

14.1 Respiratory Failure: Acuity, Evidence, and Sequencing 14.2 Heart Failure Phenotype, Acuity, and Hypertensive Combinations 14.3 Encephalopathy, Delirium, and Acute Cognitive Change

15 Chapter 15: Clinical Validation & Complication Discrepancies

15.1 Malnutrition Assessment, Severity, and Provider Documentation 15.2 Principles of a Compliant, Nonleading Query 15.3 Open-Ended, Multiple-Choice, and Yes/No Queries

16 Chapter 16: Query Mechanics & Direct Provider Collaboration

16.1 Electronic Query Systems, Templates, and Technology Oversight 16.2 Collaborating With the Appropriate Query Recipient 16.3 Capturing and Clarifying Query Responses

17 Chapter 17: Quality Indicators & Post-Discharge Clarification

17.1 Present on Admission Clarification and Selected HAC Rules 17.2 Clarifying Acuity, Cause, Site and Severity 17.3 Post-Discharge Query Opportunities and Limits 17.4 Managing Post-Discharge Query Workflows

18 Chapter 18: CDI Operational Dashboards & Quality Auditing

18.1 Departmental Metrics: Definitions, Calculations and Dashboard Use 18.2 Internal CDI Quality Audits and Reviewer Calibration 18.3 Query Trends: Understanding the Process and Provider Experience

19 Chapter 19: Benchmarking, Productivity & Data-Driven Workflow

19.1 Provider Documentation Benchmarking 19.2 External Benchmarking: CMI, Severity, Mortality and Length of Stay 19.3 Evaluating CDI Professional Performance 19.4 Using Data to Design CDI Workflow and Staffing

20 Chapter 20: Ethical CDI Standards & Regulatory Frameworks

20.1 Applying AHIMA Ethics to CDI Decisions 20.2 Responding to Financial Pressure and Upcoding Risks 20.3 Maintaining Competence and Professional Integrity

21 Chapter 21: Regulatory Compliance & Enforcement Landscapes

21.1 False Claims, Overpayments and Referral Laws 21.2 Regulatory Agencies, Review Contractors and Appeals 21.3 EHR Templates, Copied Text and Record Integrity

22 Chapter 22: Technology Risks & Second-Level Review Protocols

22.1 CAC, NLP and AI: Context, Oversight and Performance 22.2 Selecting Cases for Second-Level Review 22.3 Conducting and Closing a Second-Level Review

23 Chapter 23: Clinical Validation Queries & Risk Mitigation

23.1 Clinical Validation and Coding Clarification 23.2 Clinical Validation Queries and Denial Evidence 23.3 Detecting Leading, Biased and Unsupported Queries

24 Chapter 24: Noncompliant Query Remediation & Query Lifecycle Management

24.1 Query Remediation and Corrective Action 24.2 Query Timeframes, Aging and Measurement 24.3 Query Records, Retention and Evidence Preservation

18.2 Internal CDI Quality Audits and Reviewer Calibration

Key Takeaways Explain This Topic AI

Query practice, missed opportunities, clinical validation and coding handoff are useful audit perspectives; no metric or score alone establishes the clinical accuracy of a record.

The 2026 query brief assigns audit methods, frequency and sample selection to organizational policy; fixed national monthly chart counts and automatic-zero rubrics are not prescribed.

Targeted samples identify and explain risks but do not automatically estimate department-wide error rates; define the population, selection method, unit and denominator.

For two reviewers and nominal categories, kappa supplements raw agreement using marginal category proportions; agreement is different from correctness.

The worked 100-record table yields observed agreement 0.85, expected agreement 0.50 and kappa 0.70; descriptive labels do not mandate staffing or disciplinary actions.

Last updated: October 2026

Internal CDI Quality Audits and Reviewer Calibration

An internal audit asks whether the documentation process worked as intended and what should change when it did not. Review volume, query response and agreement percentages cannot answer that question alone. Examine the records, the actual queries, provider responses and coding decisions using the rules applicable to the encounter.

The 2026 AHIMA/ACDIS query brief , Section XIII, calls for organizational policies addressing audit methodology, frequency, sample selection and evaluation criteria. It does not prescribe a national 100-point CDI scorecard, 10–15 records per specialist each month or a universal automatic-zero scoring rule. The OIG General Compliance Program Guidance , Element 6, supports risk-informed audits and monitoring; that guidance is voluntary and does not guarantee immunity from enforcement or payer review.

Define the question and the evidence

The following are useful audit perspectives, rather than four legally required categories:

Audit perspective What to examine Error to avoid

Query practice Actual wording, sourced indicators, relevant context, response options, recipient and response record Treating a technically complete template as proof that its patient-specific use was appropriate

Missed opportunities Reviewed records with and without queries; unresolved diagnoses, specificity, procedures or timing Assuming every abnormal value or uncaptured MCC represents an error

Clinical validation Whether the documented diagnosis and query are supported in the complete clinical context Independently replacing provider judgment with one laboratory cutoff or a payer's checklist

Coding and handoff Applicable code instructions, reportability, sequencing, provider response and final abstract Requiring the final DRG to match an earlier working DRG regardless of new information

Preserve the encounter and applicable rule version. A discrepancy may reflect a true coding error, an unsupported query, new results after the concurrent review, an incomplete handoff or a reasonable issue needing further clinical clarification. Record which explanation the evidence supports rather than assigning every difference to the specialist or coder.

Evaluate substantive query integrity

Check that the query faithfully represents the available record, including relevant evidence that may oppose the suggested interpretation. Clinical indicators should retain their source and context. A multiple-choice query needs supported options and an open alternative; the 2026 brief does not require several competing diagnoses or a fixed “Other/Unable/Ruled out” trio in every case.

Leading language or pressure based on reimbursement or quality outcomes is a substantive concern requiring correction. A facility may designate such a concern as a critical failure in its own rubric. That local scoring decision is different from claiming that AHIMA mandates a zero score for the entire record. The brief also rejects treating isolated technical deviations as a rigid stand-alone denial or fraud checklist.

Example: “Please document respiratory failure to obtain an MCC” directs a preferred clinical answer for a payment purpose. The auditor should identify both defects, route remediation under policy and assess any affected documentation or coding. A provider signature does not make that wording acceptable. Conversely, the presence of just one clinically supported diagnostic option is not automatically leading when the overall question is neutral and includes the required open alternative.

For a missed-opportunity review, ask whether there was a supported, useful documentation question at the time of review. A dietitian's assessment may support provider clarification about malnutrition; it does not permit the auditor to establish the diagnosis independently. Likewise, meeting an AKI criterion can be relevant evidence, but does not make the auditor the diagnosing provider. Apply the FY2027 coding guidelines , including I.A.19, I.B.14 and Section III, alongside clinical review.

Match sample design to the purpose

Define the audit population, period, unit, selection method and exclusions before opening records. A unit might be a query, an encounter or a specific decision; the denominator must match. Include human and technology-generated queries when they are in scope.

For targeted learning , deliberately select records with a known concern, such as a recurring response-documentation problem. This is useful for understanding the problem, but its error proportion is not automatically an estimate of the whole department's performance. For population estimation , use an appropriate probability-based design with a defined sampling frame and statistical expertise when needed. Sample size depends on the question, variability, precision and intended inference, not a universal CDI chart count.

If using strata, define them so records are not unintentionally counted twice. “Queried,” “non-queried” and “high risk” can overlap: a high-risk record may also be queried. Separate those concepts or explicitly manage the overlap. An unweighted result from deliberately oversampled high-risk cases should not be presented as the overall department rate.

Hypothetical example: An auditor selects ten records specifically because a tracking report flagged possible leading wording. Four have a confirmed substantive defect. Four of ten is 40% of this selected sample . It does not establish that 40% of all departmental queries are defective. The next action may be to correct those queries and assess whether a broader, appropriately designed review is needed.

Measure reviewer agreement without confusing it with accuracy

Calibration begins with a shared decision definition and the same information available to each reviewer. Have reviewers rate the cases independently before discussing disagreements. Then reconcile the reasons, update the decision guide when appropriate and reassess consistency on a suitable set of cases. Consensus reached after discussion is useful education, but it is not independent pre-discussion agreement.

For two reviewers classifying the same records into nominal categories, Cohen's kappa can supplement raw agreement. It compares observed agreement with an expected agreement calculated from the reviewers' marginal category proportions under the specified independence model:

κ = P o − P e 1 − P e \kappa = \frac{P_o-P_e}{1-P_e} κ = 1 − P e ​ P o ​ − P e ​ ​

It does not literally eliminate all chance or prove that the shared decisions are clinically correct. Two reviewers can agree and both be wrong. Category distributions and the case mix affect interpretation. A different design, such as multiple raters or ordered severity categories, may need a different or weighted statistic rather than automatic use of ordinary two-rater kappa. See Landis and Koch's original methodological paper , Section 3.2, for the underlying agreement framework.

Worked two-reviewer example

Both reviewers independently judge whether a clarification opportunity exists in 100 hypothetical records:

  • Reviewer A B: Yes B: No Row total
  • Yes 40 10 50
  • No 5 45 50
  • Column total 45 55 100

Observed agreement is the diagonal: ( 40 + 45 ) / 100 = 0.85 (40+45)/100 = 0.85 ( 40 + 45 ) /100 = 0.85 . Reviewer A uses Yes and No equally; B uses Yes for 45% and No for 55%. Thus:

P e = ( 0.50 × 0.45 ) + ( 0.50 × 0.55 ) = 0.50 P_e=(0.50\times0.45)+(0.50\times0.55)=0.50 P e ​ = ( 0.50 × 0.45 ) + ( 0.50 × 0.55 ) = 0.50

κ = 0.85 − 0.50 1 − 0.50 = 0.70 \kappa=\frac{0.85-0.50}{1-0.50}=0.70 κ = 1 − 0.50 0.85 − 0.50 ​ = 0.70

Landis and Koch used “substantial” for 0.61–0.80, but explicitly described their divisions as arbitrary benchmarks for discussion. Those labels do not establish a mandatory CDI pass score, staffing suspension or retraining threshold. Examine the 15 disagreements, their clinical importance, the sample and the local evaluation criteria. The result describes agreement in these ratings, not validated accuracy across every service line.

Close the audit loop

Use a scorecard with defined, reproducible items, an explicit denominator and a separate description of substantive findings. A numeric average should not conceal a serious concern. Explain any critical-item rule as local policy, and distinguish not-applicable items from failures.

Give the reviewer concrete evidence, the applicable source and the required correction. Assign responsibility for record or query remediation, coding review, template changes or education as appropriate. Set follow-up timing based on risk and policy, then check whether the change worked. Neither an arbitrary quarterly score below 90% nor a single kappa label supplies a complete corrective-action plan.