On the Implementation of Computerized Adaptive Observations for Psychological Assessment

Abstract

The use of observational tools in psychological assessment has decreased in recent years, mainly due to its personnel and time costs, and researchers have not explored methodological innovations like adaptive algorithms in observational assessment. In the present study, we introduce the behavior-driven observation procedure to develop, test, and implement observational adaptive instruments. In Study 1, we use a preexisting observational checklist to evaluate nonverbal behaviors related to psychotic symptoms and to specify the adaptive algorithm’s model. We fit the model to observational data collected from 114 participants. The results support the model’s goodness of fit. In Study 2, we use the estimated model parameters to calibrate the adaptive procedure and test the algorithm for accuracy and efficiency in adaptively reconstructing 58 nonadaptively collected response patterns. The results show the algorithm’s good accuracy and efficiency, with a 40% average reduction in the number of administered items. In Study 3, we used real raters to test the adaptive checklist built with behavior-driven observation. The results indicate adequate intrarater agreement and good consistency of the observed response patterns. In conclusion, the results support the possibility of using behavior-driven observation to create accurate and affordable (in terms of resources) observational assessment tools.

Using Convolutional Neural Networks for the Assessment Research of Mental Health

“…OReilly Michelle et al analyzed 28 videos recording British children's psychology by using discourse psychology, established a rhetorical case to prove the clinical need, and believed that children's mental health is related to parents' teaching by words and deeds [ 8 ]. Since then, with the development of information technology, people began to introduce computer-aided methods to evaluate psychology, such as Heesacker Martin using computer system, and CNN proposed by some scholars to evaluate psychology [ 9 – 11 ]. The above research indicates that most of the current mental health assessment methods are mainly based on experts' experience and analysis, and there is a certain degree of subjectivity.…”

Item response theory.

“…Moreover, because of the IRT item and person invariance properties, IRT models can be used to either place items from different instruments onto a common scale, or place individuals who responded to different items onto a common scale (Boulton et al, 2019;Lee et al, 2020;Schalet et al, 2020;Victorson et al, 2019). In turn, this facilitates the analysis of differential item functioning across demographic groups (i.e., exploring whether the items measure the same latent variable in the same way across different groups (Crins et al, 2019;Flens et al, 2021;Hays et al, 2018;Liegl et al, 2020;Taple et al, 2020)), as well as the creation of "item banks" that can be administered efficiently via computerized adaptive testing (Gibbons et al, 2020;Granziol et al, 2022;Moore et al, 2019;Thomas et al, 2020).…”