Individual Differences in Computational Psychiatry: A Review of Current Challenges

Abstract

Bringing precision to the understanding and treatment of mental disorders requires instruments for studying clinically relevant individual differences. One promising approach is the development of computational assays: integrating computational models with cognitive tasks to infer latent patient-specific disease processes in brain computations. While recent years have seen many methodological advancements in computational modelling and many cross-sectional patient studies, much less attention has been paid to basic psychometric properties (reliability and construct validity) of the computational measures provided by the assays. In this review, we assess the extent of this issue by examining emerging empirical evidence. To contextualize this, we also provide a more general perspective on key developments that are needed for translating computational assays to clinical practice. Emerging evidence suggests that most computational measures show poor-to-moderate reliability and often provide little improvement over simple behavioral measures. Furthermore, behavioral and computational measures used to test computational accounts of mental disorders show a lack of convergent validity, which compromises their interpretability. Taken together, these issues pose a risk of invalidating previous findings and undermining ongoing research efforts using computational assays to study individual (and even group) differences. We suggest that cross-sectional single-task designs, which currently dominate the research landscape, are partly to blame for these problems and therefore are not suitable for solving them. Instead, reliability and construct validity need to be studied more systematically using longitudinal designs with batteries of tasks. Finally, to enable clinical applications, it will be necessary to establish predictive and longitudinal validity, and to make the assays more efficient and less burdensome.

Temporal Stability of Bayesian Belief Updating in Perceptual Decision Making

“…This provides evidence that individual differences in task-based measures of Bayesian information integration perform as a stable trait-like or phenotypic characteristic. These results support the characterisation of a latent computational measure as a means to capture potentially clinically relevant individual differences in computational psychiatry (Karvelis et al, 2022). This verification of psychometrics of cognitive-behavioural tasks should be standard practice before exploring their relationship with symptoms in clinical disorders and traitlike correlates in the general population.…”

“…Although cognitive processes will inevitably fluctuate across testing sessions, controlling for the time of day (i.e., circadian rhythm; Bedder et al, 2020) and assessing mood-related fluctuations (Eldar et al, 2018) could provide insight into state-related differences across testing sessions that might impact measurement reliability. Future research should include more sensitive investigations of state-like fluctuations, as these could provide insight into deviations from homogenous common variance models which may in fact be clinically meaningful (Karvelis et al, 2022;Sullivan-Toole et al, 2022). Despite this, testing perceptual inference in clinical settings might also lack this stringent stability and have inevitable fluctuations, suggesting that our research might hold better generalisability to clinical testing than a strictly controlled lab-based setting.…”

“…Thus, following up this study with further testing sessions would provide insight into whether practice effects impact the strategy being used (indicative of individuals' subjective likelihood uncertainty) in the likelihood-only task. Alternatively, increasing the timeframe between testing sessions might provide further insight into the trade-off between practice effects and stability in responding (Karvelis et al, 2022;Zech et al, 2022).…”

“…In turn, it also aids a phenomenological explanation for how aberrancies in the precision afforded to different types of information may characterise altered perceptual experiences (Hohwy, 2020). Whilst these aberrancies in belief updating are thought to underlie symptoms of psychopathology and trait-like correlates in non-clinical populations (Fromm et al, 2023;Gibbs-Dean et al, 2023;Karvelis et al, 2022), the assumption that individual differences in belief updating can be characterizable as a stable, trait-like phenotype is yet to be verified. This can offer valuable insight into the cognitive mechanisms underpinning behaviour (Tulver et al, 2019).…”

Neighborhood air pollution is negatively associated with neurocognitive maturation in early adolescence

“…Another is the longitudinal design of the ABCD n-back behavioral and fMRI connectivity data, which alleviates some of the concerns associated with cross-sectional studies of cognitive development. For example, when studying individual differences in development, low reliability of cognitive measures leads to a higher likelihood of spurious age-related associations in crosssectional but not longitudinal samples (Lindenberger et al, 2011;Hedge et al, 2018;Karvelis et al, 2023). Another strength of our study is the inclusion of both behavioral task and brain network measures to evaluate developmental change in neurocognitive functioning.…”