Study guide
3.4 Interobserver Agreement (IOA) Methods and Calculations
QABA Qualified Behavior Analyst QBA
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Philosophical Underpinnings and Principles of Behavior
1.1 Radical Behaviorism, Determinism & Philosophical Assumptions 1.2 Three- and Four-Term Contingencies (MOs, EOs, AOs, SDs) 1.3 Reinforcement Schedules and Stimulus Control 1.4 Verbal Operants and Stimulus Equivalence
2 Chapter 2: Behavioral Assessment
2.1 Functional Behavior Assessment Methods (Indirect, Descriptive, ABC) 2.2 Experimental Functional Analysis (Iwata Analog Conditions & Trial-Based FA) 2.3 Preference and Reinforcer Assessments (MSWO, Paired-Choice, Free Operant) 2.4 Social Validity and Behavior Intervention Plan (BIP) Design
3 Chapter 3: Experimental Design and Measurement
3.1 Single-Case Experimental Designs (Reversal, Multiple-Baseline, Alternating-Treatments, Changing-Criterion) 3.2 Visual Analysis of Data (Level, Trend, Variability, Immediacy, Overlap) 3.3 Continuous and Discontinuous Measurement Systems 3.4 Interobserver Agreement (IOA) Methods and Calculations
4 Chapter 4: Core Behavior-Change Procedures
4.1 Positive and Negative Reinforcement Tactics 4.2 Extinction and Differential Reinforcement (DRA, DRI, DRO, DRL) 4.3 Shaping, Chaining, Task Analysis & Imitation Training
5 Chapter 5: Advanced Behavior-Change Applications
5.1 Prompting, Fading, and Stimulus Control Transfer 5.2 Generalization Tactics (Stokes & Baer), Maintenance & Self-Management 5.3 Discrete Trial Teaching (DTT) and Naturalistic Teaching Strategies (NET)
6 Chapter 6: Behavior Reduction & Function-Based Replacement
6.1 Function-Based Extinction and Replacement Behavior Selection 6.2 Functional Communication Training (FCT), NCR, Setting Events & Antecedent Strategies 6.3 Restrictive Procedures, Response Cost, Time-Out, and Crisis/Safety Plans 6.4 Token Economies, Group Contingencies, Contracts & Habit Reversal
7 Chapter 7: Supervision and Training
7.1 Behavioral Skills Training (BST) for Staff and Caregivers 7.2 Treatment Integrity, Procedural Fidelity, and IOA Monitoring 7.3 Supervision Standards for ABATs and QASP-S Supervisees
8 Chapter 8: Ethics & Professional Conduct (QABA Code)
8.1 Scope of Competence, Informed Consent, and Confidentiality 8.2 Dual Relationships, Conflicts of Interest, and Billing Integrity 8.3 Mandated Reporting, Client Rights, and Cultural Responsiveness 8.4 Educational Laws: IDEA, IEP, LRE, ADA & 504 Plans 8.5 Positive Behavior Support, Person-Centered Planning & Risk-Benefit Analysis
9 Chapter 9: Autism-Specific Considerations
9.1 ASD Core Knowledge: Diagnosis, Red Flags, Comorbidity & Differential 9.2 Sensory Profiles, Restricted/Repetitive Behaviors, and Adaptation 9.3 Naturalistic Developmental Behavioral Interventions (NDBIs) and Augmentative Communication (PECS, AAC)
3.4 Interobserver Agreement (IOA) Methods and Calculations
Key Takeaways Explain This Topic AI
Interobserver Agreement (IOA) measures the degree to which two independent observers report identical values, enhancing data believability and detecting observer drift across 20% to 30% of sessions.
IOA measures agreement rather than accuracy (comparison to true values) or reliability (stability over time), meaning observers can achieve high IOA while misapplying definitions identically.
Count IOA algorithms range from coarse Total Count IOA to interval-based metrics, with Exact Count-per-Interval IOA serving as the most stringent count agreement standard.
Duration IOA algorithms evaluate overall temporal agreement via Total Duration IOA or average per-instance agreement using Mean Duration-per-Occurrence IOA.
Interval IOA requires Scored-Interval (Occurrence) IOA for low-rate behaviors (<30%) and Unscored-Interval (Nonoccurrence) IOA for high-rate behaviors (>70%) to avoid chance inflation.
Last updated: July 2026
Interobserver Agreement (IOA) Methods and Calculations
In Applied Behavior Analysis, data believability is established primarily through Interobserver Agreement (IOA) . IOA measures the degree to which two independent observers, using the same measurement system to monitor the same client during the same observation window, report identical values. High IOA indicates that the operational definition is clear and objective, observer training is adequate, and data collection is consistent.
Critical Conceptual Distinctions
On the QBA examination, candidates must clearly distinguish IOA from accuracy and reliability :
IOA vs. Accuracy : Accuracy evaluates whether observed data match the true value of the event (measured using an unrefutable gold-standard criterion, such as a calibrated video recording). IOA evaluates agreement between observers. Two observers can achieve 100% IOA while being 100% inaccurate if both misapply an operational definition identically.
IOA vs. Reliability : Reliability refers to the stability of measurement over time (e.g., whether a single observer scores a permanent product identically on two separate occasions).
Observer Drift : The gradual, unintended shift in how an observer applies an operational definition over time. Regular IOA assessments detect and correct observer drift.
Professional Standards for IOA
Frequency : IOA data must be collected across a minimum of 20% to 30% of sessions in each experimental phase.
Threshold : Acceptable IOA is generally 80% or higher (or a Cohen's Kappa coefficient of 0.60+).
Independence : Observers must collect data independently (without cues, communication, or looking at each other's data sheets).
- IOA Calculation Formulas and Worked Examples
- 1. Count / Event IOA Algorithms
- A. Total Count IOA
Calculates agreement based on the total count recorded across an entire session. Total Count IOA = ( Smaller Count Larger Count ) × 100 \text{Total Count IOA} = \left( \frac{\text{Smaller Count}}{\text{Larger Count}} \right) \times 100 Total Count IOA = ( Larger Count Smaller Count ) × 100
Worked Example : Observer 1 tallies 15 instances of hand-raising; Observer 2 tallies 20 instances. Total Count IOA = ( 15 20 ) × 100 = 75.0 % \text{Total Count IOA} = \left( \frac{15}{20} \right) \times 100 = 75.0\% Total Count IOA = ( 20 15 ) × 100 = 75.0%
Limitation : Overestimates true agreement because it does not confirm observers agreed on specific instances in time.
B. Mean Count-per-Interval IOA
Calculates the average agreement across individual intervals within a session. Mean Count-per-Interval IOA = ∑ ( Smaller Interval Count Larger Interval Count ) Total Number of Intervals × 100 \text{Mean Count-per-Interval IOA} = \frac{\sum \left( \frac{\text{Smaller Interval Count}}{\text{Larger Interval Count}} \right)}{\text{Total Number of Intervals}} \times 100 Mean Count-per-Interval IOA = Total Number of Intervals ∑ ( Larger Interval Count Smaller Interval Count ) × 100
Worked Example : Across 3 intervals: Interval 1 (Obs A = 2, Obs B = 2; Agreement = 1.0); Interval 2 (Obs A = 1, Obs B = 3; Agreement = 0.33); Interval 3 (Obs A = 4, Obs B = 5; Agreement = 0.80). Mean Count-per-Interval IOA = 1.0 + 0.33 + 0.80 3 × 100 = 2.13 3 × 100 = 71.0 % \text{Mean Count-per-Interval IOA} = \frac{1.0 + 0.33 + 0.80}{3} \times 100 = \frac{2.13}{3} \times 100 = 71.0\% Mean Count-per-Interval IOA = 3 1.0 + 0.33 + 0.80 × 100 = 3 2.13 × 100 = 71.0%
C. Exact Count-per-Interval IOA
The most conservative count metric. Only intervals with 100% exact agreement are scored as agreements. Exact Count-per-Interval IOA = ( Number of Intervals with 100% Agreement Total Number of Intervals ) × 100 \text{Exact Count-per-Interval IOA} = \left( \frac{\text{Number of Intervals with 100\% Agreement}}{\text{Total Number of Intervals}} \right) \times 100 Exact Count-per-Interval IOA = ( Total Number of Intervals Number of Intervals with 100% Agreement ) × 100
Worked Example (using the 3 intervals above): Only Interval 1 had 100% agreement (1 out of 3 intervals). Exact Count-per-Interval IOA = ( 1 3 ) × 100 = 33.3 % \text{Exact Count-per-Interval IOA} = \left( \frac{1}{3} \right) \times 100 = 33.3\% Exact Count-per-Interval IOA = ( 3 1 ) × 100 = 33.3%
2. Duration IOA Algorithms
A. Total Duration IOA Total Duration IOA = ( Shorter Duration Longer Duration ) × 100 \text{Total Duration IOA} = \left( \frac{\text{Shorter Duration}}{\text{Longer Duration}} \right) \times 100 Total Duration IOA = ( Longer Duration Shorter Duration ) × 100
Worked Example : Obs A records 45 minutes of crying; Obs B records 50 minutes. Total Duration IOA = ( 45 50 ) × 100 = 90.0 % \text{Total Duration IOA} = \left( \frac{45}{50} \right) \times 100 = 90.0\% Total Duration IOA = ( 50 45 ) × 100 = 90.0%
B. Mean Duration-per-Occurrence IOA
Calculates duration agreement for each discrete occurrence of behavior and averages them across occurrences. Mean Duration-per-Occurrence IOA = ∑ ( Shorter Duration Longer Duration ) Total Number of Responses × 100 \text{Mean Duration-per-Occurrence IOA} = \frac{\sum \left( \frac{\text{Shorter Duration}}{\text{Longer Duration}} \right)}{\text{Total Number of Responses}} \times 100 Mean Duration-per-Occurrence IOA = Total Number of Responses ∑ ( Longer Duration Shorter Duration ) × 100
3. Interval Sampling IOA Algorithms
Interval IOA algorithms are evaluated using a 2x2 contingency matrix of interval scores:
- Observer B (+) Observer B (-)
- Observer A (+) Both (+) [Agreed Occurrence] Obs A (+) / Obs B (-) [Disagreement]
- Observer A (-) Obs A (-) / Obs B (+) [Disagreement] Both (-) [Agreed Nonoccurrence]
- A. Interval-by-Interval (Total) IOA
Includes all intervals (both agreed occurrences and agreed nonoccurrences). Interval-by-Interval IOA = ( Agreed Intervals [Both + or Both -] Total Intervals ) × 100 \text{Interval-by-Interval IOA} = \left( \frac{\text{Agreed Intervals [Both + or Both -]}}{\text{Total Intervals}} \right) \times 100 Interval-by-Interval IOA = ( Total Intervals Agreed Intervals [Both + or Both -] ) × 100
Limitation : Artificially inflates agreement for very high-rate or very low-rate behaviors due to chance agreement on nonoccurrences or occurrences.
B. Scored-Interval (Occurrence) IOA
Calculates agreement based ONLY on intervals where AT LEAST ONE observer scored an occurrence (+) . Ignores intervals where both observers scored nonoccurrence (-). Scored-Interval IOA = ( Agreed Occurrence Intervals [Both +] Intervals where AT LEAST ONE observer scored + ) × 100 \text{Scored-Interval IOA} = \left( \frac{\text{Agreed Occurrence Intervals [Both +]}}{\text{Intervals where AT LEAST ONE observer scored +}} \right) \times 100 Scored-Interval IOA = ( Intervals where AT LEAST ONE observer scored + Agreed Occurrence Intervals [Both +] ) × 100
Mandatory Clinical Rule : Scored-Interval IOA must be used for LOW-RATE behaviors (occurring in <30% of intervals) to prevent agreements on nonoccurrences from masking low agreement on actual occurrences.
C. Unscored-Interval (Nonoccurrence) IOA
Calculates agreement based ONLY on intervals where AT LEAST ONE observer scored a nonoccurrence (-) . Ignores intervals where both observers scored occurrence (+). Unscored-Interval IOA = ( Agreed Nonoccurrence Intervals [Both -] Intervals where AT LEAST ONE observer scored - ) × 100 \text{Unscored-Interval IOA} = \left( \frac{\text{Agreed Nonoccurrence Intervals [Both -]}}{\text{Intervals where AT LEAST ONE observer scored -}} \right) \times 100 Unscored-Interval IOA = ( Intervals where AT LEAST ONE observer scored - Agreed Nonoccurrence Intervals [Both -] ) × 100
Mandatory Clinical Rule : Unscored-Interval IOA must be used for HIGH-RATE behaviors (occurring in >70% of intervals) to prevent agreements on occurrences from masking low agreement on nonoccurrences.