Key Points
- A second observer records the same behavior during the same stretch of session, and the two records are compared and expressed as a percentage.
- Low agreement is diagnostic - usually an ambiguous behavior definition or observer drift - and the fixes are to sharpen the definition, retrain to it, and recheck.
- Checks are scheduled across programs, observers, and times rather than incidental: checking only the easy programs with the senior tech proves little.
Interobserver Agreement (IOA) Explained
Every clinical decision in ABA rides on the data, so the data itself needs quality control. IOA provides it: a second observer records the same behavior during the same stretch of session, the two records are compared, and their agreement is expressed as a percentage.
Low agreement is diagnostic. It usually means the behavior definition is ambiguous - 'aggression' meaning different things to different techs - or that drift has crept in, with an observer's working definition slowly departing from the written one. The fixes are correspondingly concrete: sharpen the operational definition, retrain to it, and recheck.
There are several conventional ways to compute agreement - comparing total counts for the session, comparing interval by interval, or scoring agreement only on occurrences - differing mainly in how strictly they treat the moment-to-moment record. The strictness should match the stakes: data guiding a high-consequence decision deserves the stricter comparisons.
IOA is scheduled, not incidental: a portion of sessions across programs, observers, and times, so agreement is demonstrated where the data is actually produced. Checking only the easy programs with the senior tech proves little.
For practice leaders, IOA is also an audit story. Progress claims made to families and funders are only as credible as the measurement behind them, and documented agreement checks are how a practice shows its graphs mean what they say.