The LONI QC System: A Semi-Automated, Web-Based and Freely-Available Environment for the Comprehensive Quality Control of Neuroimaging Data

Abstract

Quantifying, controlling, and monitoring image quality is an essential prerequisite for ensuring the validity and reproducibility of many types of neuroimaging data analyses. Implementation of quality control (QC) procedures is the key to ensuring that neuroimaging data are of high-quality and their validity in the subsequent analyses. We introduce the QC system of the Laboratory of Neuro Imaging (LONI): a web-based system featuring a workflow for the assessment of various modality and contrast brain imaging data. The design allows users to anonymously upload imaging data to the LONI-QC system. It then computes an exhaustive set of QC metrics which aids users to perform a standardized QC by generating a range of scalar and vector statistics. These procedures are performed in parallel using a large compute cluster. Finally, the system offers an automated QC procedure for structural MRI, which can flag each QC metric as being ‘good’ or ‘bad.’ Validation using various sets of data acquired from a single scanner and from multiple sites demonstrated the reproducibility of our QC metrics, and the sensitivity and specificity of the proposed Auto QC to ‘bad’ quality images in comparison to visual inspection. To the best of our knowledge, LONI-QC is the first online QC system that uniquely supports the variety of functionality where we compute numerous QC metrics and perform visual/automated image QC of multi-contrast and multi-modal brain imaging data. The LONI-QC system has been used to assess the quality of large neuroimaging datasets acquired as part of various multi-site studies such as the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) Study and the Alzheimer’s Disease Neuroimaging Initiative (ADNI). LONI-QC’s functionality is freely available to users worldwide and its adoption by imaging researchers is likely to contribute substantially to upholding high standards of brain image data quality and to implementing these standards across the neuroimaging community.

Cortical gyrification and its relationships with molecular measures and cognition in children with the FMR1 premutation

“…Visual inspection of the MRI scans did not detect any scans with excessive motion artifacts that would affect LGI-quantifications. To further assure image quality, we obtained Euler number generated by the Freesurfer software package 34 , which summarizes topological complexity of the cerebral cortex but has been shown to correlate with manual identification of “unusable” scans 35 , as well as nine LONI Quality Control (QC) metrics that were unlikely to be affected by topological complexity 36 . The nine LONI metrics comprised (1) mean slice intensity (MSI), (2) signal to noise ratio (SNR), (3) signal variance-to-noise variance ratio (SVNR), (4) contrast-to-noise ratio (CNR), (5) contrast of variance-to-noise ratio (CVNR), (6) brain tissue contrast-to-tissue intensity variation (TCTV), (7) full-width-at-half-maximum (FWHM), (8) center of mass (CoM), and (9) coefficient of variation (COV).…”

“…However, the Euler number, reflecting topological complexity of reconstructed cortical surface, confounded with LGI. To compliment, we generated nine image QC metrics using the LONI Quality Control System 36 , including MSI, SNR, SVNR, CNR, CVNR, TCTV, FWHM, CoM, and COV. The LONI QC metrics have been validated against expert visual inspection for multi-site and multi-scanner structural MRI scans 36 .…”

“…To compliment, we generated nine image QC metrics using the LONI Quality Control System 36 , including MSI, SNR, SVNR, CNR, CVNR, TCTV, FWHM, CoM, and COV. The LONI QC metrics have been validated against expert visual inspection for multi-site and multi-scanner structural MRI scans 36 . Two additional T1 scans acquired from a 7-year-old control male and a 13-year-old PM male not included in the study because of age and excessive motion were utilized as test scans.…”

Opportunities for increased reproducibility and replicability of developmental neuroimaging

“…Likewise, standardized preprocessing pipelines may be used that provide extensive individual and group level summary reports of data quality, such as fMRIprep for functional MRI (https://fmriprep.readthedocs.io; Esteban et al, 2019) and QSIprep for diffusion weighted MRI (https://qsiprep.readthedocs.io). For extensive quality assessments of raw structural and functional MRI data, software like MRIQC (https://mriqc.readthedocs.io; Esteban et al, 2017) and LONI QC (https://qc.loni.usc.edu; Kim et al, 2019) provide a list of different image quality metrics that can be used to flag low quality scans. Decisions about the quality of processed structural image data can further be aided by the use of machine-learning output probability scores, as for instance implemented in the Qoala-T tool for FreeSurfer segmentations (Klapwijk et al, 2019).…”

The ENIGMA Stroke Recovery Working Group: Big data neuroimaging to study brain-behavior relationships after stroke

“…Data that are of poor quality are excluded from the database. More recently, to standardize this process, we have begun to use the University of Southern California's Laboratory of Neuroimaging (LONI) Quality Control system, which is a freely available, semi-automated, web-based system for quantitatively evaluating MRI image quality (Kim et al, 2019). We ensure that the behavioral data are appropriately coded according to our ENIGMA Stroke Recovery database conventions.…”