Measuring fMRI reliability with the intra-class correlation coefficient

Cáceres, Alejandro; Hall, Deanna L.; Zelaya, Fernando; Williams, Steven; Mehta, Mitul A.

doi:10.1016/j.neuroimage.2008.12.035

Cited by 226 publications

(265 citation statements)

References 23 publications

Supporting

Mentioning

244

Contrasting

Order By: Relevance

“…Reliability of brain activation was analyzed by calculating intraclass correlation coefficients (ICCs). We calculated measures of intravoxel reliability on individual contrast values for each ROI by using the ICC toolbox provided by Caceres et al (2009). The same ROIs were used as for the functional analyses.…”

Section: Methodsmentioning

confidence: 99%

A Three-Year Longitudinal Functional Magnetic Resonance Imaging Study of Performance Monitoring and Test-Retest Reliability from Childhood to Early Adulthood

Koolschijn

Schel²,

Rooij³

et al. 2011

J. Neurosci.

View full text Add to dashboard Cite

Previous cross-sectional functional magnetic resonance imaging studies have shown that performance monitoring functions continue to develop well into adolescence, associated with increased activation in brain regions important for cognitive control (prefrontal cortex, anterior cingulate cortex, and parietal cortex). To date, however, the development of performance monitoring has not yet been studied longitudinally, which leaves open the question whether changes can be detected within individuals over time. In the present study, human boys and girls, between ages 8 and 27 years, performed a child-friendly rule-switch task in the scanner on two occasions ϳ3.5 years apart. Change versus stability was examined using two methods: (1) repeated-measures analyses and (2) test-retest reliabilities of blood oxygenation level-dependent responses. Results showed that with increasing age, participants performed better on the task. The changes in neural activation associated with the processing of performance feedback were, however, more reliably correlated with changes in performance than with age. Test-retest reliability was at least fair to good for adults and adolescents, but poor to fair for the youngest age group. Substantially more variability was observed in the pattern and magnitude of children compared with adults, which may be interpreted as proxy for developmental change. Together, the results show that (1) change within individuals is variable, and more so for children than for adolescents and adults, and (2) performance is a better predictor for change in neural activation over time. These findings set the stage for studying developmental change in the perspective of multiple predictors, rather than solely by divisions based on age groups.

show abstract

Section: Methodsmentioning

confidence: 99%

A Three-Year Longitudinal Functional Magnetic Resonance Imaging Study of Performance Monitoring and Test-Retest Reliability from Childhood to Early Adulthood

Koolschijn

Schel²,

Rooij³

et al. 2011

J. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The ICC describes how members from the same group resemble each other 17,18 and has often been used to quantify the consistency of measurements made by different experts. 34,35 A high ICC shows that a feature is consistent across multiple measurements.…”

Section: Feature Consistencymentioning

confidence: 99%

“…We first sought to discover features that are robust to changes in the location and shape of the segmentation by calculating the intraclass correlation (ICC) 17,18 of features obtained from the slice selection and delineations of four readers. Because these features were robust with respect to the segmentation, we explored the possibility of obtaining similar results if we were to ask readers to merely trace a circle, which we call a "core sample," inside the tumor.…”

Section: Introductionmentioning

confidence: 99%

Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma

et al. 2015

View full text Add to dashboard Cite

Abstract. The purpose of this study is to investigate the utility of obtaining "core samples" of regions in CT volume scans for extraction of radiomic features. We asked four readers to outline tumors in three representative slices from each phase of multiphasic liver CT images taken from 29 patients (1128 segmentations) with hepatocellular carcinoma. Core samples were obtained by automatically tracing the maximal circle inscribed in the outlines. Image features describing the intensity, texture, shape, and margin were used to describe the segmented lesion. We calculated the intraclass correlation between the features extracted from the readers' segmentations and their core samples to characterize robustness to segmentation between readers, and between human-based segmentation and core sampling. We conclude that despite the high interreader variability in manually delineating the tumor (average overlap of 43% across all readers), certain features such as intensity and texture features are robust to segmentation. More importantly, this same subset of features can be obtained from the core samples, providing as much information as detailed segmentation while being simpler and faster to obtain.

show abstract

“…41 Second, instead of using an ROI-averaged amplitude or significant cluster amplitude to quantify the ICCs, Caceres et al proposed a map-wised ICCs approach, which conducts voxel-wise ICC analysis for the whole brain. 42 This approach can help discriminate the best ROIs between individuals and correlate the ICC map with the activation map since both metrics were voxel-based. We will apply this approach to study the reliability of measurements in future research.…”

Section: Future Researchmentioning

confidence: 99%

Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy–based brain imaging

Zeng

Lin

et al. 2015

J. Biomed. Opt

View full text Add to dashboard Cite

Abstract. Test-retest reliability of neuroimaging measurements is an important concern in the investigation of cognitive functions in the human brain. To date, intraclass correlation coefficients (ICCs), originally used in interrater reliability studies in behavioral sciences, have become commonly used metrics in reliability studies on neuroimaging and functional near-infrared spectroscopy (fNIRS). However, as there are six popular forms of ICC, the adequateness of the comprehensive understanding of ICCs will affect how one may appropriately select, use, and interpret ICCs toward a reliability study. We first offer a brief review and tutorial on the statistical rationale of ICCs, including their underlying analysis of variance models and technical definitions, in the context of assessment on intertest reliability. Second, we provide general guidelines on the selection and interpretation of ICCs. Third, we illustrate the proposed approach by using an actual research study to assess intertest reliability of fNIRS-based, volumetric diffuse optical tomography of brain activities stimulated by a risk decision-making protocol. Last, special issues that may arise in reliability assessment using ICCs are discussed and solutions are suggested.

show abstract

Measuring fMRI reliability with the intra-class correlation coefficient

Cited by 226 publications

References 23 publications

A Three-Year Longitudinal Functional Magnetic Resonance Imaging Study of Performance Monitoring and Test-Retest Reliability from Childhood to Early Adulthood

A Three-Year Longitudinal Functional Magnetic Resonance Imaging Study of Performance Monitoring and Test-Retest Reliability from Childhood to Early Adulthood

Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma

Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy–based brain imaging

Contact Info

Product

Resources

About