Robust group- but limited individual-level (longitudinal) reliability and insights into cross-phases response prediction of conditioned fear

Abstract

Here we follow the call to target measurement reliability as a key prerequisite for individual-level predictions in translational neuroscience by investigating i) longitudinal reliability at the individual and ii) group level, iii) cross-sectional reliability and iv) response predictability across experimental phases. 120 individuals performed a fear conditioning paradigm twice six month apart. Analyses of skin conductance responses, fear ratings and BOLD-fMRI with different data transformations and included numbers of trials were conducted. While longitudinal reliability was generally poor to moderate at the individual level, it was good for acquisition but not extinction at the group-level. Cross-sectional reliability was satisfactory. Higher responding in preceding phases predicted higher responding in subsequent experimental phases at a weak to moderate level depending on data specifications. In sum, the results suggest the feasibility of individual-level predictions for (very) short time intervals (e.g., cross-phases) while predictions for longer time intervals may be problematic.

Whole brain correlates of individual differences in skin conductance responses during discriminative fear conditioning to social cues

“…Taken together, this means that it is currently unclear to what extent our results reflect stable individual traits, as opposed to participants’ particular states at the time of measurement, which limits the interpretation of our findings. In line with Klingelhöfer-Jens et al, 2022 (Preprint) we encourage future research on individual differences in fear conditioning to explore new ways of improving the reliability of measurement.…”

“…In summary, while the use of a twin sample is a limitation of the present study, we contend that there is no reason to conclude that it invalidates the reported results. Finally, it should be noted that only a few studies to date have examined the longitudinal (test-retest) reliability of individual differences in discriminatory fear learning measured by SCR ( Cooper et al, 2022 , Preprint; Klingelhöfer-Jens et al, 2022 , Preprint). While one study found evidence of fair within-person stability in 51 participants across a 9-day period ( Cooper et al, 2022 , Preprint), another study found evidence of poor individual-level reliability in 120 participants across a 6-month period ( Klingelhöfer-Jens et al, 2022 , Preprint).…”

“…Finally, it should be noted that only a few studies to date have examined the longitudinal (test-retest) reliability of individual differences in discriminatory fear learning measured by SCR ( Cooper et al, 2022 , Preprint; Klingelhöfer-Jens et al, 2022 , Preprint). While one study found evidence of fair within-person stability in 51 participants across a 9-day period ( Cooper et al, 2022 , Preprint), another study found evidence of poor individual-level reliability in 120 participants across a 6-month period ( Klingelhöfer-Jens et al, 2022 , Preprint). Similarly, previous studies examining the individual-level reliability of fMRI BOLD responding during fear conditioning have reported low to moderate reliability ( Ridderbusch et al, 2021 ; Klingelhöfer-Jens et al, 2022 , Preprint).…”

“…While one study found evidence of fair within-person stability in 51 participants across a 9-day period ( Cooper et al, 2022 , Preprint), another study found evidence of poor individual-level reliability in 120 participants across a 6-month period ( Klingelhöfer-Jens et al, 2022 , Preprint). Similarly, previous studies examining the individual-level reliability of fMRI BOLD responding during fear conditioning have reported low to moderate reliability ( Ridderbusch et al, 2021 ; Klingelhöfer-Jens et al, 2022 , Preprint). Taken together, this means that it is currently unclear to what extent our results reflect stable individual traits, as opposed to participants’ particular states at the time of measurement, which limits the interpretation of our findings.…”

Improving the reliability of cognitive task measures: A narrative review

“…triallevel variance), and n is the number of trials. In practice, this relationship often holds [7,8] though with some exceptions [41,42]. Notably, increasing the number of task trials only benefits reliability if measurement error is random.…”