Study guide

13.3 Interobserver Agreement (IOA) Formulas, Methods, & Standards

QABA Qualified Autism Services Practitioner-Supervisor QASP-S

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Exam Overview & QASP-S Credentialing

1.1 About the QABA QASP-S Examination 1.2 Exam Strategy, Blueprint Weighting, & Study Framework

2 Chapter 2: Autism Spectrum Disorder Basics

2.1 DSM-5-TR Diagnostic Criteria & Clinical Presentation 2.2 Prevalence, Etiology, & Co-occurring Conditions 2.3 Evidence-Based Practices vs. Non-Evidence-Based Interventions 2.4 Screening, Early Identification, & Neurodiversity-Affirming Principles

3 Chapter 3: Legal, Ethical, & Professional Standards

3.1 The QABA Code of Ethical and Professional Conduct 3.2 Scope of Practice & Competence for the QASP-S 3.3 Client Rights, Dignity, Assent, & Informed Consent 3.4 Educational Law: IDEA, FAPE, LRE, IEPs, Section 504, and the ADA 3.5 Positive Behavior Supports and Tiered Prevention Frameworks 3.6 Person-Centered Planning, Self-Determination, and Collaborative Goal Setting

4 Chapter 4: Professional Boundaries, Confidentiality, & Compliance

4.1 Confidentiality, HIPAA Regulations, & Mandated Reporting 4.2 Multiple Relationships, Conflicts of Interest, & Gift Policies 4.3 Clinical Documentation, Billing Integrity, & Record Retention

5 Chapter 5: Core Principles of Applied Behavior Analysis

5.1 Philosophical Foundations & the 7 Dimensions of ABA 5.2 Respondent vs. Operant Conditioning & Behavioral Contingencies 5.3 Motivating Operations (MOs) & Discriminative Stimuli (Sd) 5.4 Positive & Negative Reinforcement Principles 5.5 The Three Levels of Scientific Understanding: Description, Prediction, and Control

6 Chapter 6: Schedules, Punishment, & Behavioral Processes

6.1 Schedules of Reinforcement & Schedule Thinning 6.2 Punishment Principles, Side Effects, & Ethical Restraints 6.3 Stimulus Control, Generalization, & Maintenance

7 Chapter 7: Antecedent Interventions

7.1 Environmental Modifications & Setting Event Management 7.2 Priming, Visual Schedules, & Social Narratives 7.3 Choice-Making, Behavioral Momentum, & High-Probability Sequences

8 Chapter 8: Function-Based Antecedent Strategies

8.1 Non-Contingent Reinforcement (NCR) Design & Implementation 8.2 Functional Communication Training (FCT): Conceptual Foundations & Protocols

9 Chapter 9: Skill Acquisition: Measurement & Foundational Instruction

9.1 Operational Definitions & Measurable Goal Formulation 9.2 Prompting Hierarchies, Prompt Types, & Prompt Fading Strategies 9.3 Discrete Trial Teaching (DTT) & Errorless Learning Procedures

10 Chapter 10: Naturalistic Instruction, Chaining, & Communication

10.1 Natural Environment Training (NET) & Incidental Teaching 10.2 Task Analysis, Shaping, & Behavior Chaining Procedures 10.3 Augmentative & Alternative Communication (AAC) & PECS Protocols 10.4 Skill Mastery Criteria, Generalization Probes, & Maintenance

11 Chapter 11: Behavior Reduction: Function & Differential Reinforcement

11.1 Behavior Intervention Plan (BIP) Essential Components 11.2 Extinction Procedures, Variations, & Side Effects 11.3 Differential Reinforcement of Alternative & Incompatible Behavior (DRA/DRI)

12 Chapter 12: Advanced Behavior Reduction & Crisis Management

12.1 Differential Reinforcement of Other & Low Rates of Behavior (DRO/DRL) 12.2 Token Economies, Contingency Contracts, & Response Cost 12.3 Crisis De-escalation, Safety Plans, & Emergency Procedures 12.4 Group Contingencies: Independent, Interdependent, and Dependent Arrangements

13 Chapter 13: Data Collection Systems & Interobserver Agreement

13.1 Continuous Measurement Procedures (Frequency, Rate, Duration, Latency, IRT) 13.2 Discontinuous Measurement Procedures (Partial, Whole, MTS, PLACHECK) 13.3 Interobserver Agreement (IOA) Formulas, Methods, & Standards

14 Chapter 14: Data Analysis & Assessment Methodologies

14.1 Visual Analysis of Single-Case Data (Level, Trend, Variability) 14.2 Single-Case Experimental Designs in Clinical Practice 14.3 Indirect & Descriptive Functional Behavior Assessment (FBA) 14.4 Functional Analysis (FA) Experimental Conditions & Safety

15 Chapter 15: Clinical Assessment, Preference, & Social Validity

15.1 Preference & Reinforcer Assessment Methodologies 15.2 Developmental & Curriculum-Based Assessments 15.3 Social Validity Assessment & Stakeholder Collaboration 15.4 Environmental Analysis and Ecological Assessment Components

16 Chapter 16: Supervision, Staff Training, & Family Collaboration

16.1 Behavioral Skills Training (BST) for Staff & Technicians 16.2 Procedural Integrity Monitoring & Competency Assessments 16.3 Delivering Performance Feedback & Managing Performance Deficits 16.4 Caregiver Training, QABA Supervision Compliance, & Burnout Prevention

13.3 Interobserver Agreement (IOA) Formulas, Methods, & Standards

Key Takeaways Explain This Topic AI

Interobserver Agreement (IOA) quantifies the degree to which two independent, simultaneous observers record the same behavioral metrics, functioning as the primary empirical standard for verifying operational definitions and detecting observer drift.

IOA evaluates data believability and measurement consistency between independent human observers; it does NOT verify data validity (whether the system measures the actual dimensional property of interest) nor data accuracy.

Continuous IOA formulas range from Total Count IOA (Smaller divided by Larger multiplied by 100) to Mean Count-per-Interval IOA and Exact Count-per-Interval IOA, with Exact Count serving as the most stringent and conservative count metric.

For interval systems, Scored-Interval IOA is ethically and methodologically mandated for low-rate behaviors (less than 30% of intervals) to eliminate artificial agreement on non-occurrences, whereas Unscored-Interval IOA is mandated for high-rate behaviors (greater than 70% of intervals) to eliminate artificial agreement on pervasive occurrences.

Professional QABA standards require IOA collection during a minimum of 20% to 33% of sessions across all baseline and intervention phases, maintaining an acceptable agreement benchmark of at least 80% (90% in research settings), with lower scores requiring systematic Behavioral Skills Training (BST) and definition refinement.

Last updated: October 2026

Interobserver Agreement (IOA) Formulas, Methods, & Standards

Exam Tip: Interobserver Agreement (IOA) calculations appear repeatedly on the QASP-S examination. You must be prepared to compute Total Count IOA , Mean Count-per-Interval IOA , Exact Count-per-Interval IOA , Total Duration IOA , Mean Duration-per-Occurrence IOA , Total Interval IOA , Scored-Interval IOA , and Unscored-Interval IOA using provided clinical datasets. Crucially, memorize the clinical rules for interval IOA: Scored-Interval IOA is mandatory for low-rate behaviors ( < 30 % < 30\% < 30% of intervals) to avoid artificial inflation from non-occurrences, while Unscored-Interval IOA is mandatory for high-rate behaviors ( > 70 % > 70\% > 70% of intervals) to avoid artificial inflation from occurrences. Remember the professional QABA standards: IOA collected in 20% to 33% of sessions with an acceptable threshold of ≥ 80 % \ge 80\% ≥ 80% .

In Applied Behavior Analysis, empirical data drive every supervisory and clinical decision—from determining whether a child has mastered an expressive communication skill to deciding whether a behavior intervention plan (BIP) should be faded or intensified. However, because behavioral data are gathered by human observers (behavior technicians, paraprofessionals, caregivers, and supervisors), the data are inherently susceptible to human measurement error, bias, fatigue, and misunderstanding.

To ensure that behavioral data are trustworthy, reliable, and scientifically robust, behavior analysts rely on Interobserver Agreement (IOA) . IOA is defined as the degree to which two or more independent, simultaneous observers report the same observed values after measuring the exact same behavioral events.

Scientific & Clinical Functions of IOA

IOA is not merely an academic exercise; it serves four vital clinical and quality-assurance functions in autism service delivery: ┌─────────────────────────────────────────┐

  • │ FOUR CORE FUNCTIONS OF IOA │
  • └────────────────────┬────────────────────┘
  • │
  • ┌───────────────────────────┬──────────────┴────────────┬───────────────────────────┐
  • ▼ ▼ ▼ ▼
  • ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
  • │ OPERATIONAL │ │ OBSERVER │ │ OBSERVER │ │ DATA │
  • │ CLARITY │ │ COMPETENCE │ │ DRIFT │ │ BELIEVABILITY │
  • │ Evaluates if │ │ Assesses if │ │ Detects if │ │ Assures funders │
  • │ the definition │ │ technicians are │ │ staff criteria │ │ & families that │
  • │ is objective, │ │ fully trained │ │ shift over time │ │ progress is real│
  • │ clear & complete│ │ and calibrated. │ │ unconsciously. │ │ not an illusion.│
  • └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  • Believability vs. Accuracy vs. Validity

The QASP-S exam tests your understanding of the precise epistemic boundaries of IOA:

Validity: A measurement system is valid if it measures the actual dimensional property of the behavior that is the true target of intervention. IOA does not assess validity. Two observers can achieve 100% agreement on a completely invalid measurement system (e.g., both measuring child weight with a scale when the clinical target is on-task attention).

Accuracy: A measurement system is accurate if the observed values match the true values of the event, determined by independent calibration instruments (e.g., millisecond-calibrated video playback). IOA does not evaluate accuracy. Two observers can share the exact same misunderstanding and achieve 100% agreement while both being completely inaccurate.

Believability / Reliability: IOA assesses believability —the extent to which independent observers arrive at consistent, repeatable measurement values. High IOA convinces researchers, clinicians, insurance payors, and parents that data variability reflects actual changes in the client's behavior rather than idiosyncratic observer differences.

Observer Drift

A primary reason for conducting ongoing IOA is detecting observer drift . Observer drift occurs when observers expand, contract, or unconsciously alter their operational definition of the target behavior over time, drifting away from the original clinical criteria established in the BIP. Without regular IOA, observer drift goes undetected, leading to false conclusions regarding client regression or progress.

Prerequisites for Valid IOA Data Collection

For an IOA assessment to be methodologically sound, five strict conditions must be fulfilled:

Identical Operational Definition: Both observers must evaluate the behavior using the exact same objective, technological definition.

Identical Measurement System: Both observers must utilize the exact same measurement system, interval lengths, and recording rules (e.g., both using 10-second Partial-Interval Recording).

Simultaneous Observation: Both observers must observe the exact same behavioral stream at the exact same time.

Completely Independent Observation: Observers must record data without communication, signaling, visual cuing, or looking at each other's data sheets. Synchronized vibrating timers or audio cues delivered through split headphones ensure interval synchronization without interpersonal signaling.

Observation of the Same Subject in the Same Context: Both observers must focus on the same learner throughout the observation window.

  • Continuous Measurement IOA Formulas & Step-by-Step Calculations
  • 1. Total Count IOA

Total Count IOA is the simplest and crudest count IOA metric. It compares the overall total counts recorded by two observers across an entire session, without evaluating whether observers agreed on specific instances or intervals. Total Count IOA = Smaller Count Larger Count × 100 \text{Total Count IOA} = \frac{\text{Smaller Count}}{\text{Larger Count}} \times 100 Total Count IOA = Larger Count Smaller Count ​ × 100

Worked Clinical Example:

Observer 1 (Technician) records 16 instances of property destruction.

Observer 2 (Supervisor) records 20 instances of property destruction. Total Count IOA = 16 20 × 100 = 80.0 % \text{Total Count IOA} = \frac{16}{20} \times 100 = 80.0\% Total Count IOA = 20 16 ​ × 100 = 80.0%

Clinical Critique: While fast and straightforward, Total Count IOA carries a severe flaw: it masks intra-session disagreements. Observer 1 could have recorded all 16 hits in the first 15 minutes, while Observer 2 recorded all 20 hits in the last 15 minutes. Despite scoring 80% agreement, they never agreed on a single actual occurrence!

2. Mean Count-per-Interval IOA

To resolve the limitation of Total Count IOA, Mean Count-per-Interval IOA divides the observation session into equal time intervals, calculates the Total Count agreement for each separate interval, and then averages these interval agreements across the entire session. Mean Count-per-Interval IOA = ∑ i = 1 n ( Smaller Count i Larger Count i ) n × 100 \text{Mean Count-per-Interval IOA} = \frac{\sum_{i=1}^n \left(\frac{\text{Smaller Count}_i}{\text{Larger Count}_i}\right)}{n} \times 100 Mean Count-per-Interval IOA = n ∑ i = 1 n ​ ( Larger Count i ​ Smaller Count i ​ ​ ) ​ × 100

(Note: Any interval where both observers record zero responses is scored as 1.00 or 100% agreement.)

Worked Clinical Example:

A 20-minute session is partitioned into four 5-minute intervals tracking vocal interruptions:

  • Interval Observer 1 Count Observer 2 Count Interval Agreement Calculation Interval Agreement
  • Interval 1 4 5 4 5 \frac{4}{5} 5 4 ​ 80.0 % 80.0\% 80.0%
  • Interval 2 2 2 2 2 \frac{2}{2} 2 2 ​ 100.0 % 100.0\% 100.0%
  • Interval 3 0 0 0 0 ⟹ 1.00 \frac{0}{0} \implies 1.00 0 0 ​ ⟹ 1.00 100.0 % 100.0\% 100.0%

Interval 4 1 3 1 3 \frac{1}{3} 3 1 ​ 33.3 % 33.3\% 33.3% Mean Count-per-Interval IOA = 80.0 % + 100.0 % + 100.0 % + 33.3 % 4 = 313.3 % 4 = 78.33 % \text{Mean Count-per-Interval IOA} = \frac{80.0\% + 100.0\% + 100.0\% + 33.3\%}{4} = \frac{313.3\%}{4} = 78.33\% Mean Count-per-Interval IOA = 4 80.0% + 100.0% + 100.0% + 33.3% ​ = 4 313.3% ​ = 78.33%

Clinical Interpretation: While Total Count IOA for this same data would be 7 10 × 100 = 70.0 % \frac{7}{10} \times 100 = 70.0\% 10 7 ​ × 100 = 70.0% , Mean Count-per-Interval evaluates agreement within localized temporal windows, offering greater insight into observational consistency.

3. Exact Count-per-Interval IOA

Exact Count-per-Interval IOA is the most conservative and rigorous count agreement metric in Applied Behavior Analysis. It evaluates the percentage of total intervals in which both observers recorded the exact identical numerical count . Exact Count-per-Interval IOA = Number of Intervals with Exact (100%) Agreement Total Number of Intervals × 100 \text{Exact Count-per-Interval IOA} = \frac{\text{Number of Intervals with Exact (100\%) Agreement}}{\text{Total Number of Intervals}} \times 100 Exact Count-per-Interval IOA = Total Number of Intervals Number of Intervals with Exact (100%) Agreement ​ × 100

Worked Clinical Example (Using the same 4 intervals above):

  • Interval 1 (4 vs. 5): Disagreement ( 0 % 0\% 0% )
  • Interval 2 (2 vs. 2): Exact Agreement ( 100 % 100\% 100% )
  • Interval 3 (0 vs. 0): Exact Agreement ( 100 % 100\% 100% )
  • Interval 4 (1 vs. 3): Disagreement ( 0 % 0\% 0% )

Exact Agreement occurred in exactly 2 out of 4 intervals. Exact Count-per-Interval IOA = 2 4 × 100 = 50.0 % \text{Exact Count-per-Interval IOA} = \frac{2}{4} \times 100 = 50.0\% Exact Count-per-Interval IOA = 4 2 ​ × 100 = 50.0%

Clinical Critique: Exact Count-per-Interval sets an exceptionally high bar. Any discrepancy, even a difference of one count (e.g., 4 vs. 5), results in zero credit for that interval. A high Exact Count-per-Interval IOA shows agreement on counts across the sampled intervals, but it does not by itself prove accuracy or validity: both observers may make the same error. Evaluate the score alongside observer training, the measurement protocol, and accuracy checks.

4. Trial-by-Trial IOA (Discrete Trial Teaching)

Trial-by-Trial IOA is used for restricted operants, discrete trial teaching (DTT), or opportunity-based tasks where responses are scored as correct ( + + + ), incorrect ( − - − ), or prompted ( P P P ). Trial-by-Trial IOA = Number of Agreed Trials Total Number of Trials Presented × 100 \text{Trial-by-Trial IOA} = \frac{\text{Number of Agreed Trials}}{\text{Total Number of Trials Presented}} \times 100 Trial-by-Trial IOA = Total Number of Trials Presented Number of Agreed Trials ​ × 100

Worked Clinical Example:

Across 10 receptive identification trials, Technician A and Supervisor B independently record learner responses:

Trials 1–7: Both record correct ( + + + ) ⟹ 7 \implies 7 ⟹ 7 agreements.

Trial 8: Tech A records correct ( + + + ); Supervisor B records prompted ( P P P ) ⟹ 1 \implies 1 ⟹ 1 disagreement.

Trial 9: Both record incorrect ( − - − ) ⟹ 1 \implies 1 ⟹ 1 agreement.

Trial 10: Tech A records incorrect ( − - − ) ; Supervisor B records prompted ( P P P ) ⟹ 1 \implies 1 ⟹ 1 disagreement.

Agreed Trials = 7 + 1 = 8 7 + 1 = 8 7 + 1 = 8 . Disagreed Trials = 2. Trial-by-Trial IOA = 8 10 × 100 = 80.0 % \text{Trial-by-Trial IOA} = \frac{8}{10} \times 100 = 80.0\% Trial-by-Trial IOA = 10 8 ​ × 100 = 80.0%

5. Total Duration IOA

Total Duration IOA compares the cumulative total duration of behavior recorded across an entire session by two independent observers. Total Duration IOA = Shorter Duration Longer Duration × 100 \text{Total Duration IOA} = \frac{\text{Shorter Duration}}{\text{Longer Duration}} \times 100 Total Duration IOA = Longer Duration Shorter Duration ​ × 100

Worked Clinical Example:

Observer 1 records a cumulative total of 42 minutes of crying during a 3-hour session.

Observer 2 records a cumulative total of 48 minutes of crying during the same session. Total Duration IOA = 42 minutes 48 minutes × 100 = 87.5 % \text{Total Duration IOA} = \frac{42 \text{ minutes}}{48 \text{ minutes}} \times 100 = 87.5\% Total Duration IOA = 48 minutes 42 minutes ​ × 100 = 87.5%

6. Mean Duration-per-Occurrence IOA

Mean Duration-per-Occurrence IOA is the most rigorous duration metric. It calculates the duration agreement for each discrete behavioral episode, sums the percentage agreements, and divides by the total number of episodes observed. Mean Duration-per-Occurrence IOA = ∑ i = 1 n ( Shorter Duration i Longer Duration i ) n × 100 \text{Mean Duration-per-Occurrence IOA} = \frac{\sum_{i=1}^n \left(\frac{\text{Shorter Duration}_i}{\text{Longer Duration}_i}\right)}{n} \times 100 Mean Duration-per-Occurrence IOA = n ∑ i = 1 n ​ ( Longer Duration i ​ Shorter Duration i ​ ​ ) ​ × 100

Worked Clinical Example:

Two observers record the duration of three discrete tantrum episodes:

Episode 1: Observer A = 6 minutes; Observer B = 8 minutes ⟹ 6 8 = 75.0 % \implies \frac{6}{8} = 75.0\% ⟹ 8 6 ​ = 75.0%

Episode 2: Observer A = 12 minutes; Observer B = 12 minutes ⟹ 12 12 = 100.0 % \implies \frac{12}{12} = 100.0\% ⟹ 12 12 ​ = 100.0%

Episode 3: Observer A = 4 minutes; Observer B = 5 minutes ⟹ 4 5 = 80.0 % \implies \frac{4}{5} = 80.0\% ⟹ 5 4 ​ = 80.0% Mean Duration-per-Occurrence IOA = 75.0 % + 100.0 % + 80.0 % 3 = 255.0 % 3 = 85.0 % \text{Mean Duration-per-Occurrence IOA} = \frac{75.0\% + 100.0\% + 80.0\%}{3} = \frac{255.0\%}{3} = 85.0\% Mean Duration-per-Occurrence IOA = 3 75.0% + 100.0% + 80.0% ​ = 3 255.0% ​ = 85.0%

Discontinuous / Interval IOA Formulas: The Inflation Traps

When evaluating time-sampling data (Partial, Whole, or MTS), selecting the correct interval IOA formula is critical. The wrong formula can artificially inflate agreement, disguising severe observer failure.

1. Interval-by-Interval (Total Interval) IOA

Interval-by-Interval IOA (also called Total Interval IOA ) assesses agreement across all intervals, regardless of whether observers scored an occurrence or a non-occurrence. Interval-by-Interval IOA = Agreed Intervals ( + / + and − / − ) Total Number of Intervals Observed × 100 \text{Interval-by-Interval IOA} = \frac{\text{Agreed Intervals } (+/+ \text{ and } -/-)}{\text{Total Number of Intervals Observed}} \times 100 Interval-by-Interval IOA = Total Number of Intervals Observed Agreed Intervals ( + / + and − / − ) ​ × 100

The Fatal Flaw / Inflation Trap of Interval-by-Interval IOA:

Interval-by-Interval IOA is heavily influenced by the baseline prevalence of the behavior :

For Low-Rate Behaviors: Observers easily achieve high agreement simply by agreeing that the behavior did not occur in most intervals.

For High-Rate Behaviors: Observers easily achieve high agreement simply by agreeing that the behavior did occur in almost every interval.

Consequently, Interval-by-Interval IOA is clinically invalid for extreme rate behaviors . Clinicians must use Scored-Interval or Unscored-Interval IOA instead.

2. Scored-Interval IOA (Occurrence IOA)

Scored-Interval IOA is ethically and methodologically mandated for LOW-RATE BEHAVIORS (behaviors occurring in ≤ 30 % \le 30\% ≤ 30% of intervals ).

The Rule: The clinician completely discards intervals where BOTH observers recorded a non-occurrence ( − / − -/- − / − ) . Agreement is calculated only across intervals in which at least one observer scored a positive occurrence . Scored-Interval IOA = Agreed Occurrence Intervals ( + / + ) Agreed Occurrence ( + / + ) + Disagreed Intervals ( + / − or − / + ) × 100 \text{Scored-Interval IOA} = \frac{\text{Agreed Occurrence Intervals } (+/+)}{\text{Agreed Occurrence } (+/+) + \text{Disagreed Intervals } (+/- \text{ or } -/+)} \times 100 Scored-Interval IOA = Agreed Occurrence ( + / + ) + Disagreed Intervals ( + / − or − / + ) Agreed Occurrence Intervals ( + / + ) ​ × 100

Comparative Proof (Low-Rate Behavior Vignette):

Two observers track low-rate severe aggression across 10 intervals. Aggression occurs rarely:

  • Interval 1 2 3 4 5 6 7 8 9 10
  • Observer 1 + + + − - − + + + − - − − - − − - − − - − − - − − - − − - −
  • Observer 2 + + + − - − − - − − - − − - − − - − − - − − - − − - − − - −

Status Agreed ( + / + ) (+/+) ( + / + ) Agreed ( − / − ) (-/-) ( − / − ) Disagreed ( + / − ) (+/-) ( + / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − ) Agreed ( − / − ) (-/-) ( − / − )

Method A: Interval-by-Interval IOA:

Both agreed on Interval 1 ( + / + +/+ + / + ), and both agreed on Intervals 2, 4, 5, 6, 7, 8, 9, 10 ( − / − -/- − / − ). That is 9 agreed intervals out of 10! Interval-by-Interval IOA = 9 10 × 100 = 90.0 % ( Falsely indicating excellent reliability! ) \text{Interval-by-Interval IOA} = \frac{9}{10} \times 100 = 90.0\% \quad (\text{Falsely indicating excellent reliability!}) Interval-by-Interval IOA = 10 9 ​ × 100 = 90.0% ( Falsely indicating excellent reliability! )

Method B: Scored-Interval IOA (Mandated for Low-Rate Behaviors):

Discard the eight ( − / − ) (-/-) ( − / − ) intervals (Intervals 2, 4, 5, 6, 7, 8, 9, 10).

Evaluate only intervals with at least one score: Interval 1 ( + / + +/+ + / + ) and Interval 3 ( + / − +/- + / − ).

Agreed occurrences = 1. Disagreements = 1. Scored-Interval IOA = 1 1 + 1 × 100 = 50.0 % ( Reveals true observational breakdown! ) \text{Scored-Interval IOA} = \frac{1}{1 + 1} \times 100 = 50.0\% \quad (\text{Reveals true observational breakdown!}) Scored-Interval IOA = 1 + 1 1 ​ × 100 = 50.0% ( Reveals true observational breakdown! )

Takeaway: Interval-by-Interval gave a misleading 90%, whereas Scored-Interval exposed that the technicians disagreed on 50% of actual aggressive episodes!

3. Unscored-Interval IOA (Non-Occurrence IOA)

Unscored-Interval IOA is ethically and methodologically mandated for HIGH-RATE BEHAVIORS (behaviors occurring in ≥ 70 % \ge 70\% ≥ 70% of intervals ).

The Rule: The clinician completely discards intervals where BOTH observers recorded an occurrence ( + / + +/+ + / + ) . Agreement is calculated only across intervals in which at least one observer scored a non-occurrence . Unscored-Interval IOA = Agreed Non-Occurrence Intervals ( − / − ) Agreed Non-Occurrence ( − / − ) + Disagreed Intervals ( + / − or − / + ) × 100 \text{Unscored-Interval IOA} = \frac{\text{Agreed Non-Occurrence Intervals } (-/-)}{\text{Agreed Non-Occurrence } (-/-) + \text{Disagreed Intervals } (+/- \text{ or } -/+)} \times 100 Unscored-Interval IOA = Agreed Non-Occurrence ( − / − ) + Disagreed Intervals ( + / − or − / + ) Agreed Non-Occurrence Intervals ( − / − ) ​ × 100

Comparative Proof (High-Rate Behavior Vignette):

Two observers track continuous vocal stereotypy across 10 intervals. Stereotypy is pervasive:

  • Interval 1 2 3 4 5 6 7 8 9 10
  • Observer 1 + + + + + + + + + + + + + + + + + + + + + + + + − - − − - −
  • Observer 2 + + + + + + + + + + + + + + + + + + + + + + + + + + + − - −

Status Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Agreed ( + / + ) (+/+) ( + / + ) Disagreed ( + / − ) (+/-) ( + / − ) Agreed ( − / − ) (-/-) ( − / − )

Method A: Interval-by-Interval IOA:

Both agreed on 8 occurrence intervals ( + / + +/+ + / + ) and 1 non-occurrence interval ( − / − -/- − / − ). Total agreed = 9 out of 10. Interval-by-Interval IOA = 9 10 × 100 = 90.0 % ( Falsely inflated by pervasive behavior! ) \text{Interval-by-Interval IOA} = \frac{9}{10} \times 100 = 90.0\% \quad (\text{Falsely inflated by pervasive behavior!}) Interval-by-Interval IOA = 10 9 ​ × 100 = 90.0% ( Falsely inflated by pervasive behavior! )

Method B: Unscored-Interval IOA (Mandated for High-Rate Behaviors):

Discard the eight ( + / + ) (+/+) ( + / + ) intervals (Intervals 1 through 8).

Evaluate only intervals with at least one non-occurrence: Interval 9 ( + / − +/- + / − ) and Interval 10 ( − / − -/- − / − ).

Agreed non-occurrences = 1. Disagreements = 1. Unscored-Interval IOA = 1 1 + 1 × 100 = 50.0 % \text{Unscored-Interval IOA} = \frac{1}{1 + 1} \times 100 = 50.0\% Unscored-Interval IOA = 1 + 1 1 ​ × 100 = 50.0%

Takeaway: Unscored-Interval IOA prevents pervasive occurrences from masking serious observer disagreement regarding when the behavior ceased.

Professional Standards for IOA Collection & Remediation

Under professional guidelines established by the Qualified Applied Behavior Analysis Credentialing Board (QABA) and the broader behavior analytic literature (e.g., Cooper, Heron, & Heward, 2020):

Frequency and Distribution of IOA Collection

Session Percentage: IOA must be collected in a minimum of 20% to 33% of sessions .

Phase Distribution: IOA must be distributed across all phases of intervention (baseline, treatment evaluation, maintenance, and generalization), not clustered conveniently in a single week.

Representative Sampling: IOA probes must be conducted across different times of day, different settings, and different behavior technicians to prevent systematic technician-specific bias.

Acceptable Benchmark: The professional clinical standard for acceptable IOA is ≥ 80 % \ge 80\% ≥ 80% (in rigorous single-case research, the standard is often set at ≥ 90 % \ge 90\% ≥ 90% ).

Systematic Supervisory Remediation Workflow for Low IOA ( < 80 % < 80\% < 80% )

When IOA falls below 80%, the QASP-S must immediately enact a structured remediation protocol: Step 1: Discrepancy Analysis ──► Step 2: Definition Audit ──► Step 3: Behavioral Skills Training ──► Step 4: Re-Calibration Probe

• Review interval sheets • Check for ambiguities • Instructions, Modeling, • Collect IOA on video probe

• Identify disagreement patterns • Add non-examples/margins • Rehearsal & Corrective Feedback • Require >= 90% before resuming

Step 1: Discrepancy Analysis: The supervisor conducts an error analysis on the raw data sheets. Did the observers disagree on specific topographies (e.g., distinguishing a loud vocalization from an actual scream), or did they experience timing misalignments (e.g., one timer was 3 seconds ahead)?

Step 2: Operational Definition Audit: Ambiguous definitions are the primary cause of low IOA. The QASP-S refines the definition, adding objective boundary conditions, explicit onset/offset criteria, and concrete examples and non-examples .

Step 3: Behavioral Skills Training (BST): The supervisor retrains technicians using the four-step BST framework: (a) verbal and written instruction, (b) video modeling of target behaviors, (c) active rehearsal scoring calibrated video clips, and (d) immediate corrective feedback.

Step 4: Re-Calibration & Mastery Criterion: The technician must achieve at least 90% IOA across three consecutive video probes before resuming independent data collection with the client.

Comprehensive Summary Table of IOA Formulas

IOA Method Mathematical Formula Primary Clinical Indication Major Clinical Advantages & Pitfalls

Total Count IOA Smaller Count Larger Count × 100 \frac{\text{Smaller Count}}{\text{Larger Count}} \times 100 Larger Count Smaller Count ​ × 100 Event recording / Frequency when intra-session timing is unimportant. Fast and simple; Major Flaw: masks timing disagreements across intervals.

Mean Count-per-Interval IOA ∑ ( Smaller i / Larger i ) n × 100 \frac{\sum (\text{Smaller}_i / \text{Larger}_i)}{n} \times 100 n ∑ ( Smaller i ​ / Larger i ​ ) ​ × 100 Interval-based frequency recording with variable rates across session. More sensitive than Total Count; evaluates consistency across temporal blocks.

Exact Count-per-Interval IOA Intervals with Exact Agreement Total Intervals × 100 \frac{\text{Intervals with Exact Agreement}}{\text{Total Intervals}} \times 100 Total Intervals Intervals with Exact Agreement ​ × 100 Highly rigorous count data verification; research and high-stakes BIPs. Most conservative count metric ; any count difference scores zero credit.

Trial-by-Trial IOA Agreed Trials Total Trials × 100 \frac{\text{Agreed Trials}}{\text{Total Trials}} \times 100 Total Trials Agreed Trials ​ × 100 Discrete Trial Teaching (DTT) and restricted operant skill acquisition. Directly evaluates opportunity-based accuracy; simple to compute in DTT.

Total Duration IOA Shorter Duration Longer Duration × 100 \frac{\text{Shorter Duration}}{\text{Longer Duration}} \times 100 Longer Duration Shorter Duration ​ × 100 Continuous duration recording across an entire observation session. Fast computation; Major Flaw: masks individual episode discrepancies.

Mean Duration-per-Occurrence IOA ∑ ( Shorter i / Longer i ) n × 100 \frac{\sum (\text{Shorter}_i / \text{Longer}_i)}{n} \times 100 n ∑ ( Shorter i ​ / Longer i ​ ) ​ × 100 Behaviors where each discrete episode duration is logged separately. Most rigorous duration metric ; evaluates duration accuracy per crisis event.

Interval-by-Interval IOA Agreed Intervals ( + / + & − / − ) Total Intervals × 100 \frac{\text{Agreed Intervals } (+/+ \text{ \& } -/-)}{\text{Total Intervals}} \times 100 Total Intervals Agreed Intervals ( + / + & − / − ) ​ × 100 Interval systems when behavior occurs at moderate rates ( 30 % 30\% 30% to 70 % 70\% 70% ). Standard interval metric; Major Flaw: artificially inflated by extreme rates.

Scored-Interval IOA Agreed ( + / + ) Agreed ( + / + ) + Disagreements × 100 \frac{\text{Agreed } (+/+)}{\text{Agreed } (+/+) + \text{Disagreements}} \times 100 Agreed ( + / + ) + Disagreements Agreed ( + / + ) ​ × 100 MANDATORY for Low-Rate Behaviors ( < 30 % < 30\% < 30% of intervals). Discards ( − / − ) (-/-) ( − / − ) intervals; prevents artificial agreement inflation on non-occurrences.

Unscored-Interval IOA Agreed ( − / − ) Agreed ( − / − ) + Disagreements × 100 \frac{\text{Agreed } (-/-)}{\text{Agreed } (-/-) + \text{Disagreements}} \times 100 Agreed ( − / − ) + Disagreements Agreed ( − / − ) ​ × 100 MANDATORY for High-Rate Behaviors ( > 70 % > 70\% > 70% of intervals). Discards ( + / + ) (+/+) ( + / + ) intervals; prevents artificial agreement inflation on pervasive occurrences.

Interobserver Agreement Formula Selection Algorithm

Loading diagram... Interobserver Agreement (IOA) Formula Selection Decision Tree