Computerized adaptive testing to screen children for emotional and behavioral problems by preventive child healthcare

Abstract

Background: Questionnaires to detect emotional and behavioral problems (EBP) in Preventive Child Healthcare (PCH) should be short which potentially affects validity and reliability. Simulation studies have shown that Computerized Adaptive Testing (CAT) could overcome these weaknesses. We studied the applicability (using the measures participation rate, satisfaction, and efficiency) and the validity of CAT in routine PCH practice. Methods: We analyzed data on 461 children aged 10-11 years (response 41%), who were assessed during routine well-child examinations by PCH professionals. Before the visit, parents completed the CAT and the Child Behavior Checklist (CBCL). Satisfaction was measured by parent-and PCH professional-report. Efficiency of the CAT procedure was measured as number of items needed to assess whether a child has serious problems or not. Its validity was assessed using the CBCL as the criterion. Results: Parents and PCH professionals rated the CAT on average as good. The procedure required at average 16 items to assess whether a child has serious problems or not. Agreement of scores on the CAT scales with corresponding CBCL scales was high (range of Spearman correlations 0.59-0.72). Area Under Curves (AUC) were high (range: 0.95-0.97) for the Psycat total, externalizing, and hyperactivity scales using corresponding CBCL scale scores as criterion. For the Psycat internalizing scale the AUC was somewhat lower but still high (0.86). Conclusions: CAT is a valid procedure for the identification of emotional and behavioral problems in children aged 10-11 years. It may support the efficient and accurate identification of children with overall, and potentially also specific, emotional and behavioral problems in routine PCH.

Developing a Web-Based App to Assess Mental Health Difficulties in Secondary School Pupils: Qualitative User-Centered Design Study

“…Although CAT has previously been applied to facilitate mental health assessments in different patient groups [26,27,49,50], the use of CAT to assess children and young people's mental health is relatively new [30]. In contrast, early prototyping, the involvement of users, and continuous iteration are key principles of a user-centered design approach [40,51], which in recent years has become more prominent in the development of mental health web apps [52][53][54][55][56].…”

“…Adaptive tests for screening a range of mental health difficulties, including anxiety, depression, substance misuse, and suicidality, are now available in cloud-based environments and are being introduced in emergency departments, primary and secondary health care settings, student health clinics, child welfare, and justice systems [26]. Numerous simulation and evaluation studies have confirmed that CAT is a highly accurate method for assessing mental health difficulties in different populations [26][27][28][29][30]. However, very little is known about the feasibility and user perceptions of technology-based mental health assessments, particularly in the younger population.…”

A Review of Standardised Assessment Development Procedure and Algorithms for Computer Adaptive Testing: Applications and Relevance for Fourth Industrial Revolution

“…Findings revealed that adaptive-based administrations impacted achievement without compromising testing validity over time. Also,Kane et al (2020) andTheunissen et al (2020) applied CAT in developing more concise Patient-Reported Outcome Measures (PROM) using the Veterans RAND 12 Item Health Survey (VR-12) deployed adaptively to decrease patients' question burden, a 33% decrease. Therefore, the CAT model was termed efficient in improving PROM as well as patient experience.CAT has also been applied to large-scale language testing programmes for placement purposes such as The Quick Placement Test (QPT), Test of English as a Foreign Language (TOEFL), Computerised Oral Proficiency Instrument (COPI) and Basic English Skills Test (BEST) Plus.…”