Study guide
4.1 Operational Definitions and Direct Continuous Measurement
Board Certified Assistant Behavior Analyst BCaBA
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the BCaBA Exam: 2026 Format, Fees, Blueprint, and Study Plan
2 Chapter 1: Behaviorism and Philosophical Foundations
1.1 Goals of Behavior Analysis & Foundational Assumptions 1.2 Radical Behaviorism vs. Methodological Behaviorism & Rejection of Mentalism 1.3 Seven Dimensions of Applied Behavior Analysis & Branches of Behavior Analysis
3 Chapter 2: Core Behavioral Principles & Contingencies
2.1 Behavior, Response Classes, Stimulus Classes, and Conditioning Paradigms 2.2 Positive and Negative Reinforcement & Schedules of Reinforcement 2.3 Positive and Negative Punishment & Unwanted Effects of Aversives 2.4 Unconditioned, Conditioned, and Generalized Reinforcers and Punishers
4 Chapter 3: Operant Mechanisms & Complex Principles
3.1 Operant and Respondent Extinction Processes 3.2 Motivating Operations vs. Discriminative Stimuli 3.3 Stimulus Control, Equivalence, and Elementary Verbal Operants 3.4 Rule-Governed vs. Contingency-Shaped Behavior; Instructions, Rules, and Modeling
5 Chapter 4: Measurement Systems & Interobserver Agreement
4.1 Operational Definitions and Direct Continuous Measurement 4.2 Discontinuous Measurement Systems & Measurement Artifacts 4.3 Measurement Quality: Validity, Accuracy, Reliability, and IOA Calculations
6 Chapter 5: Data Display & Visual Analysis
5.1 Graphing Conventions: Line Graphs, Cumulative Records, and Standard Celeration Charts 5.2 Visual Analysis of Graphed Data: Level, Trend, Variability, and Immediacy of Effect 5.3 Baseline Logic and Data-Based Clinical Decision Making
7 Chapter 6: Experimental Design & Internal Validity
6.1 Characteristics of Single-Case Experimental Designs & Experimental Control 6.2 Core Single-Case Designs: Reversal, Multiple Baseline, Alternating Treatments, and Changing Criterion 6.3 Threats to Internal Validity & Social/External Validity
8 Chapter 7: Ethics in Behavior Analysis: Core Responsibilities & Client Rights
7.1 BACB Ethics Code Structure, Core Principles, and Professional Competence 7.2 Responsibility to Clients: Confidentiality, Informed Consent, and Client Dignity 7.3 Multiple Relationships, Conflicts of Interest, and Gift-Giving Prohibitions 7.4 Risks of Unethical Conduct, Public Statements, and Transitioning Clients and Supervisors
9 Chapter 8: Ethics in Assessment, Intervention, & Regulatory Compliance
8.1 Ethical Standards in Behavior Assessment and Intervention Selection 8.2 Mandated Reporting, Legal Obligations, and Documentation/Billing Integrity 8.3 Cultural Responsiveness, Interprofessional Collaboration, and Disciplinary Reporting 8.4 Interpersonal Skills for Professional Relationships and Ongoing Professional Development
10 Chapter 9: Behavioral Assessment & Functional Assessment
9.1 Indirect Assessment Methods and Review of Records 9.2 Descriptive Functional Behavior Assessment: Direct Observation and Data Collection 9.3 Experimental Functional Analysis (Analog FA) and Specialized Variations 9.4 Preference and Reinforcer Assessments & Social Validity 9.5 When Services Are Warranted, Skills Assessment, and Contextual Fit
11 Chapter 10: Foundational Behavior-Change Procedures
10.1 Differential Reinforcement Procedures (DRA, DRI, DRO, DRL, DRH) 10.2 Prompting Strategies, Prompt Hierarchies, and Systematic Prompt Fading 10.3 Shaping (Successive Approximations) and Chaining Procedures 10.4 Token Economies, Contingency Contracts, and Group Contingencies
12 Chapter 11: Antecedent & Function-Based Interventions
11.1 Functional Communication Training (FCT) and Noncontingent Reinforcement (NCR) 11.2 Antecedent Manipulations: High-Probability Sequences, Premack Principle, and Choice Architecture 11.3 Programming for Generalization and Maintenance: Stokes and Baer (1977) Tactics
13 Chapter 12: Teaching New Skills: Acquisition Arrangements, Discriminations, Verbal Behavior, and Self-Management
12.1 Discrete-Trial, Free-Operant, and Naturalistic Teaching Arrangements 12.2 Teaching Simple and Conditional Discriminations 12.3 Teaching Elementary Verbal Operants: Mands, Echoics, Tacts, and Intraverbals 12.4 Procedures That Promote Emergent Relations and Generative Performance 12.5 Self-Management Procedures
14 Chapter 13: Intervention Development, Fidelity, & Monitoring
13.1 Writing Socially Valid, Observable, and Measurable Behavioral Goals 13.2 Designing Behavior Support Plans: Function-Based Multi-Component Architecture 13.3 Treatment Integrity Monitoring, Fidelity Measurement, and Data-Based Plan Revision 13.4 Managing Behavioral Relapse, Transition Planning, and Systematic Discharge Criteria
15 Chapter 14: Supervisory Roles, Behavioral Skills Training, & BACB Requirements
14.1 The BCaBA Role: Practicing Under Ongoing BCBA Supervision and Professional Boundaries 14.2 Behavioral Skills Training (BST) for Direct Care Staff and RBTs 14.3 Objective Performance Monitoring, Performance Feedback, and Supervision Documentation 14.4 Promoting Cultural Humility, Equity, and Ethical Supervision in Behavioral Healthcare 14.5 Effective Supervisory Relationships: Practices, Expectations, and Supervision Goals 14.6 Performance Management, Performance Diagnostics (PDC-HS), and Evaluating Supervision
4.1 Operational Definitions and Direct Continuous Measurement
Key Takeaways Explain This Topic AI
Operational definitions must satisfy Hawkins and Dobes (1977) criteria—Objective, Clear, and Complete—delineating explicit boundary conditions, inclusionary criteria, and exclusionary non-examples.
Direct measures observe the behavior itself as it occurs, indirect measures rely on reports or proxies (interviews, rating scales), and permanent-product measures record the lasting results of behavior (e.g., completed worksheets) after it occurs.
Direct continuous measurement quantifies three fundamental dimensional quantities of behavior: Repeatability (count, rate/frequency), Temporal Extent (duration), and Temporal Locus (response latency, interresponse time).
Rate (count per unit of observation time) is the most sensitive measure for free-operant behaviors, but introduces severe artificial ceiling effects when misapplied to restricted-operant tasks or discrete trials.
Temporal locus measures include response latency (delay from S D S^D S D to initiation) and interresponse time (IRT, inverse to rate: R a t e ≈ 1 / I R T Rate \approx 1/IRT R a t e ≈ 1/ I RT ), while derivative measures like percentage lack absolute dimensional properties and distort small-sample data.
Last updated: October 2026
Foundations of Applied Behavioral Measurement
Direct, objective measurement is the empirical cornerstone of applied behavior analysis. Science cannot describe, predict, or demonstrate experimental control over phenomena that cannot be precisely quantified. In clinical and educational environments, assistant behavior analysts rely on measurement to establish baseline levels of performance, evaluate intervention efficacy, determine whether behavioral goals have been met, and maintain accountability to clients, families, and funding agencies.
Measurement in behavior analysis is not an administrative afterthought; it is an active clinical process. A flawed measurement system produces flawed data, leading to erroneous clinical decisions—such as continuing an ineffective intervention or prematurely terminating a necessary behavior plan. To establish an effective measurement system, the behavior analyst must first define the target behavior with exacting operational precision and then select the dimensional quantity that most accurately captures the clinical reality of the response class.
Operational Definitions: The Hawkins & Dobes (1977) Framework
Before any behavioral event can be counted, timed, or sampled, observers must agree completely on what constitutes an instance of that behavior. An operational definition provides an explicit, unambiguous description of the target behavior and its environmental boundaries.
In their seminal work, Hawkins and Dobes (1977) established that a clinically valid operational definition must satisfy three indispensable criteria: it must be objective , clear , and complete . [ HAWKINS & DOBES (1977) CRITERIA ]
- |
- +----------------------------+----------------------------+
- | | |
- v v v
- [ OBJECTIVE ] [ CLEAR ] [ COMPLETE ]
- Observable physical Unambiguous, readable, Delineates boundary
- characteristics; no passes 'Stranger Test'; conditions, inclusions,
- mentalistic inferences leaves no room for doubt and explicit non-examples
- 1. Objective
The definition must refer exclusively to observable characteristics of the behavior and immediate environmental events. It must completely exclude unobservable mentalistic states, internal drives, hypothetical constructs, and emotional attributions. Phrases such as "acts out because he is angry," "shows disrespect," or "has a bad attitude" fail the objective criterion because internal feelings cannot be physically seen or calibrated by an independent observer. An objective definition describes the exact physical movements emitted by the organism (e.g., "strikes open hand against peer's arm with sufficient force to produce an audible slap").
2. Clear
The definition must be unambiguous, direct, and readily understood by a novice observer. A clear definition allows an individual with minimal behavioral background to read the description and immediately execute the measurement protocol without hesitation. A classic heuristic used to assess clarity is the "Stranger Test" : if an independent observer (a "stranger" to the client) can read the definition and record the behavior with high agreement compared to experienced staff, the definition passes the test of clarity. Furthermore, definitions must satisfy Ogden Lindsley's "Dead Man's Test" : if a dead man can do it (e.g., "sitting quietly" or "remaining still"), it is not a behavior and must be re-framed as an active motor response.
3. Complete
The definition must delineate the precise boundary conditions of the response class. It must specify what constitutes an onset (initiation) and offset (termination) of the response, provide exhaustive inclusionary criteria, and outline explicit non-examples (contrast classes). Non-examples prevent observers from scoring topographically similar responses that do not belong to the target response class. For example, if defining "property destruction," a non-example might state: "Dropping a pencil from the desk accidentally or gently placing a book on the floor does not count as property destruction."
Topographical vs. Functional Response Definitions
Behavior analysts categorize operational definitions into two overarching classes: topographical definitions and functional definitions .
Topographical Definitions
A topographical definition defines behavior strictly by its physical form, shape, motor properties, or anatomical appearance. The definition specifies the movements involved, the bodily effectors used, and the physical parameters of the action, without regard to the consequence maintaining the response.
When to Use Topographical Definitions:
In early baseline assessment before a functional behavior assessment (FBA) or functional analysis (FA) has identified the maintaining variables.
When the target behavior produces inconsistent, diffuse, or variable environmental outcomes across occurrences.
When different topographies belonging to the same broad functional class pose vastly different safety risks requiring distinct clinical containment protocols (e.g., separating head-banging against brick walls from hand-mouthing, even if both are maintained by automatic reinforcement).
Functional Definitions
A functional definition defines behavior by its common functional outcome—the specific effect the response produces on the physical or social environment, or the consequence that maintains it. All topographically distinct behaviors that access the same reinforcer are grouped into a single operant response class .
When to Use Functional Definitions:
Whenever the functional reinforcer maintaining the behavior has been identified.
When multiple distinct motor actions produce the identical environmental consequence (e.g., screaming, throwing chairs, and dropping to the floor all reliably produce escape from academic tasks).
When the clinician seeks the most parsimonious, efficient measurement system that directly informs function-based treatment.
Feature Topographical Definition Functional Definition
Primary Basis Physical form, bodily mechanics, and motor appearance Environmental consequence and maintaining function
Focus How the behavior looks What the behavior does / achieves
Inclusion Standard Restricted to specified motor actions regardless of outcome Includes all motor actions that produce the targeted consequence
Clinical Parsimony Often requires separate tracking systems for multiple forms Highly parsimonious; tracks entire response classes concurrently
Exemplar "Client raises closed fist and strikes peer's torso from a distance of at least 6 inches." "Client engages in any motor or vocal behavior that results in escape or postponement of task demands."
Fundamental Dimensional Quantities of Behavior
Behavior is a physical phenomenon that occurs in time and space. As articulated by Johnston and Pennypacker (1980, 1993), all direct measures of behavior are derived from three fundamental dimensional quantities: Repeatability , Temporal Extent , and Temporal Locus . [ THREE FUNDAMENTAL DIMENSIONS OF BEHAVIOR ]
- |
- +-----------------------------+-----------------------------+
- | | |
- v v v
- [ REPEATABILITY ] [ TEMPORAL EXTENT ] [ TEMPORAL LOCUS ]
- Behavior recurs repeatedly Behavior occupies time Behavior occurs at a specific
- over time (Countability) (Duration) point in time relative to events
- | | |
- Count, Rate, Total Duration, Response Latency,
- Celeration Duration-per-Occurrence Interresponse Time (IRT)
- 1. Repeatability (Countability)
Repeatability refers to the fact that a behavioral event can recur repeatedly through time. The organism can emit the same response class multiple times. Dimensional measures of repeatability include count , rate/frequency , and celeration .
2. Temporal Extent
Temporal extent refers to the fact that every behavioral event occupies a specific duration of time. The response has an identifiable onset and offset, and the physical movement persists across an elapsed temporal interval. The primary dimensional measure of temporal extent is duration .
3. Temporal Locus
Temporal locus refers to the fact that a behavioral event occurs at a specific point in time relative to other environmental events or preceding responses. It captures the exact temporal relationship between the behavior and antecedent stimuli or surrounding responses. Dimensional measures of temporal locus include response latency and interresponse time (IRT) .
Topography and Magnitude
Task C.1 also names magnitude . Two further dimensions matter when form or force is the clinical issue:
Topography is the physical form of the response (for example, letter formation in handwriting or the posture used during a transfer). It is usually measured against a checklist of correct form.
Magnitude is the force or intensity of the response (for example, voice volume in decibels or the force of a hit). It is measured with an instrument or a rating anchored to observable criteria.
Direct, Indirect, and Permanent-Product Measures
Task C.3 asks you to distinguish three ways of obtaining data:
Measure What is recorded Example Main caution
Direct The behavior itself, observed as it occurs (live or on video) An observer counts hits during a session Requires an observer present at the right time
Indirect Something other than the behavior of interest, such as a report about it A parent rates tantrum frequency on a questionnaire Lower validity; depends on memory and bias
Permanent product A lasting outcome the behavior leaves behind Number of math problems completed correctly; units assembled The product must reflect the target behavior and the right person
Permanent-product measurement is efficient because the observer does not need to watch the behavior happen; products can be scored later and rescored for IOA. Use it only when (a) each occurrence of the behavior reliably produces the same product, (b) the product cannot be produced by something else (for example, a sibling completing the worksheet), and (c) the product is available when the data are recorded. Direct measurement is preferred whenever feasible because it is the most valid view of the behavior; indirect measures are best used to generate hypotheses, not to evaluate treatment.
Direct Continuous Measurement Procedures in Clinical Practice
Continuous measurement procedures record every single occurrence of the target behavior during an observation period. Because they capture every instance from onset to offset, continuous measurement systems do not rely on sampling intervals and do not introduce systematic estimation errors. The primary continuous measurement procedures are detailed below.
1. Count
Count is a simple tally of the absolute number of times a target response is emitted during an observation window. Count is calculated by incrementing an integer tally (using a hand tally counter, digital clicker, or tally sheet) each time the response criteria are met.
Clinical Indications: Count is appropriate only when the observation period is of fixed, identical duration across all sessions, the behavior has clear discrete start and stop points, and response rates are moderate.
Limitations & Pitfalls: Count provides unstandardized data if observation times vary. For example, recording 15 instances of aggression during a 30-minute session represents a drastically different behavioral intensity (0.5 responses/min) than 15 instances recorded during a 6-hour school day (0.04 responses/min). Presenting raw count across varying session lengths violates basic empirical measurement principles.
2. Rate / Frequency
Rate (often termed frequency in applied literature) is defined as the count of responses per unit of observation time. Rate combines repeatability with time, standardizing measurement across observation sessions of varying lengths.
Rate = Total Count of Responses Total Duration of Observation Period \text{Rate} = \frac{\text{Total Count of Responses}}{\text{Total Duration of Observation Period}} Rate = Total Duration of Observation Period Total Count of Responses
Standard Units: Responses per minute, responses per hour, or responses per day.
Clinical Utility: Rate is widely regarded as the most sensitive, meaningful dimensional measure of free-operant behavior. Free operants are behaviors that can be emitted at any time, have discrete onset and offset boundaries, require minimal time to complete, and do not depend on external trial presentations (e.g., mands, vocal tics, hair-pulling, self-injurious hits).
The Restricted-Operant Trap and Ceiling Effects
A critical board-level distinction exists between free operants and restricted operants (discrete trials). In restricted-operant arrangements, the client can emit the response only in the presence of an explicit antecedent prompt or trial presentation delivered by a practitioner (e.g., presenting a flashcard and stating "What is this?").
Critical Exam Trap: Never use rate to measure performance during discrete trial training (DTT) or restricted-operant tasks. When responses are tethered to teacher-delivered trials, the observed rate is constrained by the speed of the instructor's trial delivery, not the client's behavioral repertoire. If a teacher presents 10 trials across 20 minutes, the client cannot emit more than 10 responses, creating an artificial ceiling effect . For restricted operants, percentage of opportunities or trials-to-criterion must be utilized.
3. Duration
Duration is the cumulative amount of time an individual engages in a target behavior from onset to offset. In clinical practice, duration is recorded in two distinct formats:
Total Duration: The cumulative sum of time an individual spends engaged in the behavior across the entire observation session. Calculated by summing the elapsed duration of all individual episodes. Total duration is often reported as total minutes or converted to a percentage of total observation time ( Total Duration Total Session Time × 100 % \frac{\text{Total Duration}}{\text{Total Session Time}} \times 100\% Total Session Time Total Duration × 100% ).
Duration-per-Occurrence: The exact duration of each discrete episode recorded separately. This reveals both the central tendency (mean duration) and the range of episode lengths.
Clinical Indications: Duration is indicated when the behavior occurs continuously over extended periods (e.g., crying, tantrums, sleep disturbances, staying seated, academic task engagement, or motor stereotypy). It is also necessary when the primary clinical goal is to alter the length of time the behavior persists rather than how frequently it initiates.
Operational Precision Requirement: Duration measurement requires explicit, objective definitions of response onset (e.g., "crying begins when tears fall or vocal screaming exceeds conversational volume for 3 consecutive seconds") and offset (e.g., "crying terminates when no screaming or sobbing occurs for 30 consecutive seconds"). Without offset thresholds, observers will record wildly divergent episode durations.
4. Response Latency
Response latency is the amount of time that elapses between the presentation of an antecedent stimulus (such as a discriminative stimulus S D S^D S D , instructional command, or environmental signal) and the physical initiation of the response. [ Antecedent Stimulus Presented ] -------------------> [ Response Initiated ]
|<----------------- RESPONSE LATENCY ----------------->|
Latency vs. Duration: Latency measures the time from the antecedent trigger to the start of the action; duration measures the time from the start of the action to its end.
Clinical Indications: Latency is indicated when the clinical objective is to decrease or increase the delay between a command and action:
Decreasing Latency: Improving instructional compliance, decreasing processing delay, or quickening evacuation responses during fire alarms.
Increasing Latency: Addressing impulsivity, rapid answering before questions are completed, or teaching delay-of-gratification waiting repertoires.
5. Interresponse Time (IRT)
Interresponse Time (IRT) is the amount of time elapsed between the termination (offset) of one response and the initiation (onset) of the very next consecutive response within the same response class. [ Response 1 Offset ] --------------------------------> [ Response 2 Onset ]
|<----------------- INTERRESPONSE TIME (IRT) ---------->|
The Inverse Mathematical Relationship with Rate: IRT and rate are functionally and mathematically interdependent. When responses are emitted rapidly in close succession, IRT is very short and rate is high. When responses are spaced far apart in time, IRT is long and rate is low.
Rate ≈ 1 Mean IRT \text{Rate} \approx \frac{1}{\text{Mean IRT}} Rate ≈ Mean IRT 1
Clinical Indications:
Differential Reinforcement of Low Rates (DRL): When reducing a behavior that is acceptable at moderate rates (e.g., asking questions in class, clearing throat) but disruptive at high rates, the clinician reinforces responses only when the elapsed IRT exceeds a predetermined criterion (e.g., IRT ≥ 5 minutes \ge 5\text{ minutes} ≥ 5 minutes ).
Differential Reinforcement of High Rates (DRH): When increasing fluency or rapid responding, reinforcement is delivered only when the IRT is shorter than a specified limit.
Pacing Programs: Lengthening IRT to remediate rapid food ingestion (eating pacing protocols to prevent choking) or shortening IRT to accelerate sluggish transitions.
Derivative Measures: Percentages and Trials-to-Criterion
Behavior analysts frequently transform continuous dimensional quantities into derivative measures . While widely utilized, derivative measures lack fundamental dimensional units of their own and must be interpreted with caution.
1. Percentage
A percentage is a derived ratio formed by combining two identical dimensional quantities (such as counts or durations), dividing the observed value by the total opportunities, and multiplying by 100. Most commonly, it expresses the proportion of correct or independent responses: Percentage = Number of Correct Responses Total Opportunities Presented × 100 % \text{Percentage} = \frac{\text{Number of Correct Responses}}{\text{Total Opportunities Presented}} \times 100\% Percentage = Total Opportunities Presented Number of Correct Responses × 100% .
Critical Pitfalls of Percentage:
No Temporal Dimension: Percentage conveys nothing about the speed, latency, or rate of responding. A student who scores 90% correct while completing 10 math problems over 60 minutes appears identical on paper to a student who scores 90% correct in 2 minutes.
Ceiling and Floor Artifacts: Percentages are artificially bounded between 0% and 100%, masking significant gains in fluency once 100% accuracy is reached.
Small-Denominator Distortion: Calculating percentages over small sample sizes ( < 20 <20 < 20 or < 30 <30 < 30 trials) produces extreme volatility. For example, if only 4 trials are presented, a single error swings the calculated score from 100% to 75%, misrepresenting the true stability of the learner's repertoire.
2. Trials-to-Criterion
Trials-to-criterion measures the total number of response opportunities, instructional presentations, or practice sessions required for a learner to achieve a predetermined, objective mastery criterion (e.g., "number of discrete trials required to achieve 3 consecutive sessions at 90% accuracy").
Clinical Indications: Trials-to-criterion is used to compare the relative efficiency of competing instructional methods, curriculum sequences, or prompting hierarchies. It directly quantifies learning rate across skill acquisition programs.
Continuous Measurement Dimensions Matrix
The following matrix summarizes the fundamental dimensions, core measurable properties, mathematical formulations, clinical indications, and anti-patterns across continuous measurement systems:
Dimension Core Measurable Property Calculation / Unit Clinical Indications Limitations / Anti-Patterns
Count Repeatability; tally of discrete response emissions Total integer count (e.g., 14 hits) Fixed observation periods; discrete responses with clear start/stop Completely invalid for comparing sessions of differing durations
Rate / Frequency Repeatability combined with time Count Observation Time \frac{\text{Count}}{\text{Observation Time}} Observation Time Count (e.g., 2.5 responses/min) Free-operant behaviors; measuring fluency, strength of stimulus control Misleading ceiling effects when applied to restricted-operant/DTT tasks
Total Duration Temporal Extent; cumulative time behavior persists ∑ ( Durations of all episodes ) \sum (\text{Durations of all episodes}) ∑ ( Durations of all episodes ) (e.g., 24 total minutes) Extended continuous behaviors (tantrums, crying, sustained on-task) Masks whether total represents one long episode or many brief episodes
Duration-per-Occurrence Temporal Extent; length of each individual episode Individual episode times; mean & range (e.g., mean 42s, range 10-120s) Evaluating shifts in episode intensity and endurance over time Demands continuous stopwatch monitoring with explicit onset/offset rules
Response Latency Temporal Locus; elapsed delay from stimulus to action Time from antecedent S D S^D S D to initiation of response (e.g., 6.2s) Compliance latency; processing delay; impulsivity; fire drill readiness Confounding latency with duration; measuring when behavior ends rather than starts
Interresponse Time (IRT) Temporal Locus; elapsed time between consecutive responses Time from offset of R 1 R_1 R 1 to onset of R 2 R_2 R 2 (e.g., mean IRT = 18s) DRL/DRH shaping schedules; pacing eating; conversational pauses Difficult to record manually for high-rate bursts; inverse to rate
Percentage Derivative; proportional ratio of identical quantities Responses Opportunities × 100 % \frac{\text{Responses}}{\text{Opportunities}} \times 100\% Opportunities Responses × 100% (e.g., 85%) Discrete trial training; restricted operants; accuracy assessments Distorted by small denominators ( < 20 <20 < 20 trials); lacks temporal rate info
Trials-to-Criterion Derivative; learning speed and instructional efficiency Total trials/sessions required to reach mastery (e.g., 42 trials) Comparing teaching procedures; evaluating curriculum effectiveness Post-hoc metric; does not evaluate moment-to-moment behavioral performance