A Novel System for Supporting Autism Diagnosis Using Home Videos: Iterative Development and Evaluation of System Design

Abstract

BackgroundObserving behavior in the natural environment is valuable to obtain an accurate and comprehensive assessment of a child’s behavior, but in practice it is limited to in-clinic observation. Research shows significant time lag between when parents first become concerned and when the child is finally diagnosed with autism. This lag can delay early interventions that have been shown to improve developmental outcomes.ObjectiveTo develop and evaluate the design of an asynchronous system that allows parents to easily collect clinically valid in-home videos of their child’s behavior and supports diagnosticians in completing diagnostic assessment of autism.MethodsFirst, interviews were conducted with 11 clinicians and 6 families to solicit feedback from stakeholders about the system concept. Next, the system was iteratively designed, informed by experiences of families using it in a controlled home-like experimental setting and a participatory design process involving domain experts. Finally, in-field evaluation of the system design was conducted with 5 families of children (4 with previous autism diagnosis and 1 child typically developing) and 3 diagnosticians. For each family, 2 diagnosticians, blind to the child’s previous diagnostic status, independently completed an autism diagnosis via our system. We compared the outcome of the assessment between the 2 diagnosticians, and between each diagnostician and the child’s previous diagnostic status.ResultsThe system that resulted through the iterative design process includes (1) NODA smartCapture, a mobile phone-based application for parents to record prescribed video evidence at home; and (2) NODA Connect, a Web portal for diagnosticians to direct in-home video collection, access developmental history, and conduct an assessment by linking evidence of behaviors tagged in the videos to the Diagnostic and Statistical Manual of Mental Disorders criteria. Applying clinical judgment, the diagnostician concludes a diagnostic outcome. During field evaluation, without prior training, parents easily (average rating of 4 on a 5-point scale) used the system to record video evidence. Across all in-home video evidence recorded during field evaluation, 96% (26/27) were judged as clinically useful, for performing an autism diagnosis. For 4 children (3 with autism and 1 typically developing), both diagnosticians independently arrived at the correct diagnostic status (autism versus typical). Overall, in 91% of assessments (10/11) via NODA Connect, diagnosticians confidently (average rating 4.5 on a 5-point scale) concluded a diagnostic outcome that matched with the child’s previous diagnostic status.ConclusionsThe in-field evaluation demonstrated that the system’s design enabled parents to easily record clinically valid evidence of their child’s behavior, and diagnosticians to complete a diagnostic assessment. These results shed light on the potential for appropriately designed telehealth technology to support clinical assessments using in-home video captured by families. This asse...

Investigating the accuracy of a novel telehealth diagnostic approach for autism spectrum disorder.

“…Still, clinical judgment is often based on a two-way exchange of information between patients and clinicians rather than a single opportunity to share information. Although not utilized in this initial validation study, the NODA system also includes a feature to allow raters to request additional information from families (e.g., re-recording a scenario with additional social presses from the parent), which shows up in the form of an alert within the family’s NODA application (see Nazneen et al, 2015, for more details). This feature provides an additional opportunity for the rater to solicit clinically relevant information to clarify the nature of the child’s behavior and perhaps improve the accuracy and confidence of clinical judgment.…”

“…interact with your child playfully; say your child’s name to get their attention; ask your child where something is in the room; give your child time to initiate or respond; point at something and direct your child’s attention to it). Pilot studies demonstrated that these instructions improved the clinical utility of the videos (Nazneen et al, 2015). To avoid predisposing parents toward collecting examples of behaviors that indicate ASD (e.g.…”

“…Pilot studies have demonstrated parents’ ability to collect clinically relevant videos of child behavior in the home and to share relevant developmental history information, as well as diagnosticians’ ability to identify sufficient behavioral examples in the videos to satisfy diagnostic criteria for ASD (Smith, Oberleitner, Treulich, McIntosh, & Melmed, 2009; Nazneen et al, 2015). Still, comparison of the resulting diagnostic outcomes to a gold-standard, in-person assessment (IPA) has not yet been reported.…”

Whittling Down the Wait Time

“…47 One study has even designed a diagnostic tool using home videos recorded by parents. 48 Specifically, a program in Kansas uses professional teams in the patient’s local community to perform standardized assessments, which are then presented to the family and a university-based medical center team that makes the final diagnostic assessment and provides recommendations via telemedicine. 42 As academic hospital centers rapidly expand their technological resources, telemedicine provides a modern venue for increasing accessibility to specialized services for children with developmental concerns.…”

Patient-Centered Methods for Designing and Developing Health Information Communication Technologies: A Systematic Review

“…Most studies (39%) included £15 participants. 13,21,[34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53] Fourteen studies had >15 but <25 participants, [54][55][56][57][58][59][60][61][62][63][64][65][66][67] and 14 studies had between 26 and 59 participants. [68][69][70][71][72][73][74][75][76][77][78][79][80][81] Three studies described the population but did not include the number of participants.…”