Co-design and qualitative validation of animated assessment item content for a child-reported digital distress screener

Abstract

Purpose The Interactive Child Distress Screener ( ICDS ) is a novel, digital screening tool that is currently under development and aims to broadly assess self-reported emotional and behavioural distress in children aged five to 11 years. This study implemented a generative participatory codesign and iterative refinement process to qualitatively validate the content of 30 animated assessment items developed for the ICDS by assessing their acceptability and accuracy from the child's perspective. Methods The participants ( N = 62) were five to 11-year-old children. Individual interviews were conducted with each child to determine acceptability and validity of animated items and facilitate the co-design refinement process of the animated assessment items. Results Twenty-two out of 30 (73%) items met ≥80% satisfaction and accuracy consensus in their original format, six items (20%) required one round of refinement before meeting consensus, and two items (7%) required two rounds of refinements. Combined acceptability of animated items was high, ranging from 4.1 to 5 out of 5 across all items. Conclusion Participants were able to accurately identify and understand socio-emotional and behavioural constructs when depicted as animated items. Acceptability was high, even in first iterations when accuracy of understanding required refinement. This study highlighted the importance and benefits of iterative participatory design methodology in ensuring assessment items developed for children are understood, accepted and likely to be effective in obtaining accurate self-report.

Adaptive Web Interfaces for Inclusive Services: A Study on Designing for Persons with Disabilities

“…The adopted methodology for this project is participatory design [10,23]. Participatory design has been successfully employed in various internet intervention projects [9,25,31]. This approach actively involves the individuals whom the outcome of the design process will directly impact.…”

“…In addition to autonomy, mitigating social isolation plays a vital role in the mental wellbeing of individuals with disabilities. Research studies have consistently shown that fostering a sense of belonging and reducing feelings of isolation significantly improve the mental well-being of individuals with disabilities [6,31].…”

Psychometric Evaluation of a New Digitally Animated Child Self-Report Assessment Instrument: The Interactive Child Distress Screener

“…The ICDS was co-developed with more than 100 children (aged 4–12 years) and has been described in previous feasibility and development studies (March et al, 2018; Zieschank, Day et al, 2021; Zieschank, Machin et al, 2021). In the first of these development studies, child participants discussed, defined, and modeled audiovisual and behavioral exemplars for each of the contrasting emotional and behavioral construct pairs as the first step of the item content co-design process.…”

“…The resulting shared interpretations formed the narrative framework for and subsequent creation of 30 prototype animated ICDS assessment items (Zieschank, Machin et al, 2021). In the second of these studies, child participants' understanding of the prototypes was evaluated throughout iterative co-design cycles of animation testing, analysis, and refinement (Zieschank, Day et al, 2021). The content validity of the animated items was supported when participants as young as 5 years could accurately identify the intended emotional and behavioral constructs depicted in each animation.…”