Evaluation of the accuracy, reliability, efficiency, and acceptability of fast forwarding to score problem behavior

Abstract

Technological advances have allowed professionals to obtain extended recordings of caregiver–client interactions in natural settings, but scoring recorded video at normal speed to identify instances of low‐rate problem behavior is impractical in terms of scoring time. Fast forwarding is a continuous measurement system in which all seconds of an observation are viewed at a speed faster than normal. In Study 1, we evaluated whether three groups of five observers could discriminate problem behavior at three fast‐forwarding speeds across 10‐min observations. We analyzed the efficiency of using fast forwarding compared to continuous scoring, and interobserver agreement across the fast‐forwarding speeds. In Study 2, we compared the accuracy, efficiency, and social acceptability of fast forwarding (3.5x) and momentary time sampling (3.5 s) across 90‐min observations. Results support the use of 3.5x fast forwarding as a viable measurement system of improving the practicality of scoring problem behavior from video.

Procedures and Accuracy of Discontinuous Measurement of Problem Behavior in Common Practice of Applied Behavior Analysis

“…frequently the subject of experimental analysis (Jessel, Metras, Hanley, Jessel, & Ingvarsson, in press;Machado, Luczynski, & Hood, 2019;Saini, Fisher, & Retzlaff, 2018). A practitioner's choices about how direct-observation data are collected impact other important decisions, such as the determination of the function of problem behavior in a functional analysis and decisions about when to implement or change interventions.…”

Computer‐based training to teach observers to accurately score problem behavior using fast forwarding at 5x normal speed

“…The featured caregiver–client dyad and the location in which recording took place were assigned across vignettes based on confederate and researcher schedules. The vignettes were scripted (programmed) and recorded using the same procedures to create original vignettes as described by Machado et al (2019). The form of each occurrence of a target topography could vary across dyads, given that the form met the operational definition (e.g., we ensured all confederate clients kicked an object, but we did not control which object they kicked).…”

“…In a preliminary analysis of fast forwarding, Linscheid et al (1984) demonstrated that fast forwarding (12.0x) was reliable, efficient, and allowed for accurate data collection for some but not all target responses with five observers across two children. Recently, Machado et al (2019) evaluated whether three groups of five observers could accurately score discrete problem behavior at 3.5x, 4.0x, and 5.0x across 10‐min observations (Study 1). They then compared the accuracy, efficiency, and acceptability of fast‐forwarding (3.5x), momentary‐time‐sampling (3.5 s), and normal‐speed measurement systems across 90‐min observations (Study 2).…”

“…This difference may be considered minimal, but the 19% improvement in efficiency at 10.0x would enhance the practicality of scoring problem behavior when used across longer and repeated observations. However, at half this speed (i.e., 5.0x), Machado et al (2019) found that observers reliably omitted (missed) instances of problem behavior, resulting in an unacceptable level of scoring error and suggesting that correctly discriminating the occurrence of problem behavior at faster speeds is unlikely without specialized training. One solution could be to use training procedures that include exposure to multiple examples of problem behavior and repeated practice scoring with immediate feedback at the target fast‐forwarding speed.…”

“…To extend Linscheid et al (1984) and Machado et al (2019), we evaluated a computer‐based training program designed to teach observers to score problem behavior at 5.0x, the fastest speed evaluated by Machado et al, with minimal or no omission errors. The choice to evaluate a computer‐based program was made for two reasons.…”