Site effects how-to & when: an overview of retrospective techniques to accommodate site effects in multi-site neuroimaging analyses

Abstract

Site differences, or systematic differences in feature distributions across multiple data-acquisition sites, are a known source of heterogeneity that may adversely affect large-scale meta- and mega-analyses of independently collected neuroimaging data. They influence nearly all multi-site imaging modalities and biomarkers, and methods to compensate for them can improve reliability and generalizability in the analysis of genetics, omics, and clinical data. The origins of statistical site effects are complex and involve both technical differences (scanner vendor, head coil, acquisition parameters, imaging processing) and differences in sample characteristics (inclusion/exclusion criteria, sample size, ancestry) between sites. In an age of expanding international consortium research, there is a growing need to disentangle technical site effects from sample characteristics of interest. Numerous statistical and machine learning methods have been developed to control for, model, or attenuate site effects - yet to date, no comprehensive review has discussed the benefits and drawbacks of each for different use cases. Here, we provide an overview of the different existing statistical and machine learning methods developed to remove unwanted site effects from independently collected neuroimaging samples. We focus on linear mixed effect models, the ComBat technique and its variants, adjustments based on image quality metrics, normative modelling, and deep learning approaches such as generative adversarial networks. For each method, we outline the statistical foundation and summarize strengths and weaknesses, including their assumptions and conditions of use. We provide information on software availability and comment on the ease of use and the applicability of these methods to different types of data. We discuss validation and comparative reports, mention caveats and provide guidance on when to use each method, depending on context and specific research questions.

Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection

“…Existing retrospective data harmonization techniques have shown promise in removing site-related variance from different studies to allow for such pooled analyses. Most harmonization methods fall into two broad categories: (1) harmonization of image-derived features using statistical properties of the distribution, for example, ComBat (Chen et al, 2022;Pomponio et al, 2020;Zhao et al, 2019); (2) harmonization the of the results of specific tasks, such as a regional segmentation, disease classification, or age prediction. That is, to ensure that images from various sources produce consistent and reliable outcomes when used in the same tasks.…”

“…The main drawback of the first category of harmonization techniques is that it requires many statistical assumptions that may be difficult to satisfy. An extensive review of the first category of harmonization techniques was performed by Bayer et al (2022). The second category, which seeks to circumvent the statistical assumption pitfalls by avoiding the harmonization of datasets directly but focusing on the MR image output, is largely composed of deep learning‐based approaches, namely domain adaptation techniques (Guan et al, 2021; Wang, Chaudhari, & Davatzikos, 2022).…”

Investigating Dopaminergic Abnormalities in Psychosis with Normative Modelling and Multisite Molecular Neuroimaging

“…However, these numbers cannot be feasibly achieved in single molecular neuroimaging studies due to the substantial costs of PET and SPECT scans (up to 10 times higher than MRI scans) and ethical issues related to the use of radioactive tracers for research purposes. Nevertheless, recent developments in the molecular neuroimaging community, including a greater willingness to share data 29,30 and the establishment of international consortia (e.g., ENIGMA 31 ), along with the development of effective harmonisation techniques for neuroimaging data [32][33][34][35] , have paved the way for the use of normative modelling in molecular neuroimaging.…”

Style Transfer Generative Adversarial Networks to Harmonize Multi-Site MRI to a Single Reference Image to Avoid Over-Correction

“…The main drawback of the first category of harmonization techniques is that it requires many statistical assumptions that may be difficult to satisfy. An extensive review of the first category of harmonization techniques was performed by Bayer et al, [8].The second category, which seeks to circumvent the statistical assumption pitfalls by avoiding the harmonization of datasets directly but focusing on the task output, is largely composed of deep learning-based approaches, namely domain adaptation techniques [9,10]. Domain transfer learning and domain adversarial learning have been applied for MRI harmonization [11].…”