No abstract

Background/ObjectivesApparent diffusion coefficient (ADC) and signal intensity (SI) measurements play an increasing role in magnetic resonance imaging (MRI) of monoclonal plasma cell disorders. The purpose of this study was to assess interrater variability, repeatability, and reproducibility of ADC and SI measurements from bone marrow (BM) under variation of MRI protocols and scanners.Patients and MethodsFifty-five patients with suspected or confirmed monoclonal plasma cell disorder were prospectively included in this institutional review board–approved study and underwent several measurements after the standard clinical whole-body MR scan, including repeated scan after repositioning, scan with a second MRI protocol, scan at a second 1.5 T scanner with a harmonized MRI protocol, and scan at a 3 T scanner. For T1-weighted, T2-weighted STIR, B800 images, and ADC maps, regions of interest were placed in the BM of the iliac crest and sacral bone, and in muscle tissue for image normalization. Bland-Altman plots were constructed, and absolute bias, relative bias to mean, limits of agreement, and coefficients of variation were calculated.ResultsInterrater variability and repeatability experiments showed a maximal relative bias of −0.077 and a maximal coefficient of variation of 16.2% for all sequences. Although the deviations at the second 1.5 T scanner with harmonized MRI protocol to the first 1.5 T scanner showed a maximal relative bias of 0.124 for all sequences, the variation of the MRI protocol and scan at the 3 T scanner led to large relative biases of up to −0.357 and −0.526, respectively. When comparing the 3 T scanner to the 1.5 T scanner, normalization to muscle reduced the bias of T1-weighted and T2-weighted sequences, but not of ADC maps.ConclusionsThe MRI scanners with identical field strength and harmonized MRI protocols can provide relatively stable quantitative measurements of BM ADC and SI. Deviations in MRI field strength and MRI protocol should be avoided when applying ADC cutoff values, which were established at other scanners or when performing multicentric imaging trials.

Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labelling instructions are key. Despite their importance, their optimization remains largely unexplored. Here we present a systematic study of labelling instructions and their impact on annotation quality in the field. Through comprehensive examination of professional practice and international competitions registered at the Medical Image Computing and Computer Assisted Intervention Society, the largest international society in the biomedical imaging field, we uncovered a discrepancy between annotators’ needs for labelling instructions and their current quality and availability. On the basis of an analysis of 14,040 images annotated by 156 annotators from four professional annotation companies and 708 Amazon Mechanical Turk crowdworkers using instructions with different information density levels, we further found that including exemplary images substantially boosts annotation performance compared with text-only descriptions, while solely extending text descriptions does not. Finally, professional annotators constantly outperform Amazon Mechanical Turk crowdworkers. Our study raises awareness for the need of quality standards in biomedical image analysis labelling instructions.