Recently published guidelines recommend the validation of models for medical image processing against variations of image acquisition parameters. This work presents a novel concept for systematic tests that investigate a model's performance to MRI acquisition shifts in brain scans of multiple sclerosis patients. Contrast changes of T2-weighted FLAIR images related to changes of two scan parameters (TE, TI) are simulated using artificial data. These images were created from 114 brain MRI, normal tissue segmentation masks and expert MS lesion masks (NAMIC, OpenMS, MSSEG2, ISBI15, LC08). Contrast changes were simulated by using the FLAIR signal equation. The experiments evaluate the F1 scores of models trained on images based on different uniform and non-uniform acquisition parameters when tested on typical acquisition parameter shifts. The acquisition parameter variations of the experiments are based on guidelines for quality assurance and the analysis of protocols and images from different institutions and open datasets. The dependence of the F1 score on both parameters could be approximated by linear functions with R 2 scores of 0.94 to 0.98. Thus, linear functions can be used to model the dependence of the F1 score on the influencing factors, thereby allowing for the derivation of a "save" range of image acquisition parameters to meet a desired performance metric. The segmentation model was more sensitive to changes in TE compared to TI. The maximum lesion F1 loss when applying the models to out-of-distribution data ranged from 0.04 to 0.36 and was significant in all the experiments, even when the model was trained on data representing scans of different contrast (p<0.01). This underlines the need for testing segmentation models against acquisition shifts.

No abstract