A Survey of Crowdsourcing in Medical Image Analysis

Ørting, Silas Nyboe; Doyle, Andrew; Hirth, Matthias; Hilten, Arno van; Inel, Oana; Madan, Christopher R.; Mavridis, Panagiotis; Spiers, Helen; Cheplygina, Veronika

doi:10.15346/hc.v7i1.1

Cited by 36 publications

(36 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, there has been interest in crowd sourcing a large number of novice decisions for medical diagnosis or to create labeled data sets to train artificial intelligence systems (Alialy et al, 2018;Dissanayake et al, 2019;Kamar et al, 2012;Mozafari et al, 2014;Ørting et al, 2020).…”

Section: Harnessing the Wisdom Of The Confident Crowd In Medical Image Decision-makingmentioning

confidence: 99%

“…Third, there is recent interest in using novices to assist with medical image diagnosis. As mentioned above, this opens up the possibility of crowdsourcing large numbers of untrained individuals to perform simple diagnostic tasks (Alialy et al, 2018;Dissanayake et al, 2019;Ørting et al, 2020).…”

Section: Harnessing the Wisdom Of The Confident Crowd In Medical Image Decision-makingmentioning

confidence: 99%

See 1 more Smart Citation

Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Hasan¹,

Eichbaum²,

Seegmiller³

et al. 2021

Preprint

View full text Add to dashboard Cite

Improving the accuracy of medical image interpretation is critical to improving the diagnosis of many diseases. Using both novices (undergraduates) and experts (medical professionals), we investigated methods for improving the accuracy of a single decision maker and a group of decision makers by aggregating repeated decisions in different ways. Participants made classification decisions (cancerous versus non-cancerous) and confidence judgments on a series of cell images, viewing and classifying each image twice. We first examined whether it is possible to improve individual-level performance by using the maximum confidence slating algorithm (Koriat, 2012b), which leverages metacognitive ability by using the most confident response for an image as the ‘final response’. We find maximum confidence slating improves individual classification accuracy for both novices and experts. Building on these results, we show that aggregation algorithms based on confidence weighting scale to larger groups of participants, dramatically improving diagnostic accuracy, with the performance of groups of novices reaching that of individual experts. In sum, we find that repeated decision making and confidence weighting can be a valuable way to improve accuracy in medical image decision-making and that these techniques can be used in conjunction with each other.

show abstract

Section: Harnessing the Wisdom Of The Confident Crowd In Medical Image Decision-makingmentioning

confidence: 99%

Section: Harnessing the Wisdom Of The Confident Crowd In Medical Image Decision-makingmentioning

confidence: 99%

Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Hasan¹,

Eichbaum²,

Seegmiller³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The segmentation, identification and analysis of EM cellular images can be performed through manual processes [ 52 , 53 , 54 ], which can be distributed as citizen science where an army of non-experts [ 55 , 56 ] are recruited to provide non-expert human annotation, segmentation or classification through web-based interfaces (e.g., (Accessed on 28 May 2021)) [ 57 ]. Alternatively, computational approaches with traditional algorithms or deep learning approaches have been proposed to detect neuronal membrane and for mitosis detection in breast cancer [ 58 ], mitochondria [ 59 , 60 ], synapses [ 61 ] and proteins [ 62 ].…”

Section: Introductionmentioning

confidence: 99%

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

2021

View full text Add to dashboard Cite

In this work, an unsupervised volumetric semantic instance segmentation of the plasma membrane of HeLa cells as observed with serial block face scanning electron microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8192 × 8192 pixels each. The background was used to create a distance map, which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., the largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2000 × 2000 × 300 voxels that were treated as cell instances. Then, for each of these volumes, the nucleus was segmented, and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions of interest previously selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and the Jaccard similarity Index (JI): nucleus: JI =0.9665, AC =0.9975, cell including nucleus JI =0.8711, AC =0.9655, cell excluding nucleus JI =0.8094, AC =0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background. In samples with tightly packed cells, this may not be available. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data were released openly through GitHub, Zenodo and EMPIAR.

show abstract

“…Segmentation, identification and analysis of EM cellular images can be performed through manual processes [39][40][41], which can be distributed as citizen science where an army of non-experts [42,43] are recruited to provide non-expert human annotation, segmentation or classification through web-based interfaces (e.g. https://www.zooniverse.org/projects/h-spiers/etch-a-cell) [44].…”

Section: Introductionmentioning

confidence: 99%

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

Karabağ

Jones

Reyes-Aldasoro

2021

Preprint

View full text Add to dashboard Cite

In this work, the unsupervised volumetric semantic segmentation of the plasma membrane of HeLa cells as observed with Serial Block Face Scanning Electron Microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8, 192 x 8, 192 pixels each. The background was used to create a distance map which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2, 000 x 2, 000 x 300 voxels that were treated as cell instances. Then, for each of these volumes the nucleus was segmented and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and Jaccard Index (JI): Nucleus: JI = 0.9665, AC= 0.9975, Cell and Nucleus JI = 0.8711, AC = 0.9655, Cell only JI = 0.8094, AC = 0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background, as in cases of tightly packed cells. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data are released openly through GitHub, Zenodo and EMPIAR.

show abstract

A Survey of Crowdsourcing in Medical Image Analysis

Cited by 36 publications

References 57 publications

Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

Contact Info

Product

Resources

About