Test-retest reliability of functional connectivity in depressed adolescents

Abstract

The test-retest reliability of fMRI functional connectivity is a key factor in the identification of reproducible biomarkers for psychiatric illness. Low reliability limits the observable effect size of brain-behavior associations. Despite this important connection to clinical applications of fMRI, few studies have explored reliability in populations with psychiatric illnesses or across age groups. We investigate the test-retest reliability of functional connectivity in a longitudinal cohort of adolescents with and without major depressive disorder (MDD). Measuring reliability is complex and several metrics exist that can offer unique perspectives: for example, univariate metrics capture reliability of a single connection at a time while multivariate metrics reflect stability of the entire connectome. We compare a widely used univariate metric, intraclass correlation coefficient (ICC), and two multivariate metrics, fingerprinting and discriminability. Depressed adolescents were more reliable than healthy adolescents at the univariate level (0.34 > 0.24; Wilcoxon rank-sum:p< .001), and both groups had poor average ICCs (<0.4). Multivariate reliability was high in both groups: fingerprinting (FIHV= 0.53; FIMDD= 0.45; Poisson(1) testp< .001) and discriminability were above chance (DiscrHV= 0.75;DiscrMDD= 0.76; 500-fold permutation testp< .01). Reliability was not associated with symptoms or medication, suggesting that there is not a strong relationship between depression and reliability. These findings support the shift towards multivariate analysis for improved power and reliability.

Reliability of brain metrics derived from a Time-Domain Functional Near-Infrared Spectroscopy System

“…To summarize, quantifying the test-retest reliability of the metrics derived from a neuroimaging system is critical. If a system were to be used for diagnostic purposes and identification of neuropsychiatric and/or neurodegenerative biomarkers, a lack of adequate reliability could obscure any true underlying signals 1 . The reliability of signals recorded from Kernel Flow2-on a variety of tasks, at multiple time points, and using different headsets-suggests its potential to fill a need for scalable multi-device and even multi-site neuroimaging in the scientific and medical communities.…”

“…This multiple linear regression problem is then solved using a least-squares solution that includes prewhitening the time courses with an autoregressive model. The design matrix, X, included a combination of block-level and standard nuisance regressors such as (1) time course of the task blocks (square waves convolved with a a canonical hemodynamic response function for each block type); (2) drift regressors; and (3) low frequency cosine terms. GLM Contrasts (test statistics) and their corresponding significance (p-value) were then computed between conditions of interest via statistical tests (t-tests) on the fitted coefficients.…”

“…In order for a metric to have therapeutic or diagnostic utility, it must be stable in the absence of structural or functional changes in an individual. Under this condition, any change in the metric can be attributed to a bona fide underlying change in the brain-behavior axis, which may reflect disease progression or response to clinical intervention 1,2 .…”

“…On the other hand, an observed lack of reliability does not readily invalidate a measure; it may simply indicate that the function of interest was not properly held constant across measurements, or may represent a valid change. While the bulk of non-invasive functional neuroimaging studies has sought to validate derived neural metrics, a growing body of literature has explored the TRR of these measurements in neuropsychiatric conditions such as depression 1,2 , cognitive decline 4,5 , pain 6 , stroke 7 , aging 8 , and ADHD 9 . These studies are helpful to establish the current standard of TRR across common noninvasive neuroimaging modalities (functional Magnetic Resonance Imaging, fMRI; electroencephalography, EEG; and functional Near-Infrared Spectroscopy, fNIRS).…”

Connectome caricatures: removing large-amplitude co-activation patterns in resting-state fMRI emphasizes individual differences

“…Caricatured connectomes have greater multivariate but lower univariate reliability. While often going hand in hand, increasing multivariate does not guarantee an increase in univariate reliability 38 . They are distinct.…”