Accurate Simulation of Operating System Updates in Neuroimaging Using Monte-Carlo Arithmetic

Salari, Ali; Chatelain, Yohan; Kiar, Gregory; Glatard, Tristan

doi:10.1007/978-3-030-87735-4_2

Cited by 4 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the experiment using FL, to evaluate changes in precision for intermediate files we use the significant digit metrics. Similarly to what is done in [17], we measure results precision as the number of significant bits among result samples obtained with the "fuzzified" pipeline, as s = − log 2 σ µ , where σ and µ are respectively the observed cross-sample standard deviation and average.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Reproducibility of Tumor Segmentation Outcomes with a Deep Learning Model

Ligneris,

Bonnet,

Chatelain

et al. 2023

2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)

Self Cite

View full text Add to dashboard Cite

In the last few years, there has been a growing awareness of reproducibility concerns in many areas of science. In this work, our goal is to evaluate the reproducibility of tumor segmentation outcomes produced with a deep segmentation model when MRI images are pre-processed (i) with two different versions of the same pre-processing pipeline, and (ii) by introducing numerical perturbations that mimic executions on different environments. Results show that these two variability sources can lead to important variations of segmentation outcomes: Dice can go as low as 0.59 and Hausdorff distance as high as 84.75. Moreover, both cases show a similar range of values, suggesting that the underlying causes for instability may be numerical stability. This work can be used as a benchmark to improve the numerical stability of the pipeline.

show abstract

Section: Discussionmentioning

confidence: 99%

“…One important element of computational environments is operating systems (OS). "Fuzzy libmath" (FL) is a framework that uses Monte-Carlo arithmetic to simulate the variability induced by OS changes [17]. Here we evaluate the difference in results triggered by the change in OS as simulated with FL.…”

Section: Methodsmentioning

confidence: 99%