Superhuman cell death detection with biomarker-optimized neural networks

Linsley, Jeremy W.; Linsley, Drew; Lamstein, Josh; Ryan, Gennadi; Shah, Kevan; Castello, Nicholas A.; Oza, Viral; Kalra, Jaslin; Wang, Shijie; Tokuno, Zachary; Javaherian, Ashkan; Serre, T.; Finkbeiner, Steve

doi:10.1126/sciadv.abf8142

Cited by 12 publications

(15 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Discussionmentioning

confidence: 99%

“…There is growing consensus that DNNs can automate tasks on biomedical imaging data, achieving performance that matches or exceeds human experts [30, 40]. It is also becoming clear that these DNN achievements are due to visual strategies that are not necessarily aligned with those used by human experts [26, 28, 30]. When humans and machines use different strategies to solve tasks it can be a positive development, with the chance to reveal new insights into biology, and generate testable hypotheses for understanding the development of disease31.…”

Section: Discussionmentioning

confidence: 99%

“…Model saliency maps are noisy estimates of feature importance, and subsequent advancements in explainable artificial intelligence have involved modifications of the gradient to reduce noise in these maps. A particularly effective approach for biomedical imaging [30] is Guided Gradient-weighted Class Activation Mapping (Grad-CAM [31]), which we use here to identify morphological features that drove DNN molecular profiling decisions. We specifically focus on interpreting Shufflenet's decision making, because it performed best of the models we tested at detecting KRAS mutations on WSIs.…”

Section: Task 1: Dnns Trained For Molecular Profiling Are Vulnerable ...mentioning

confidence: 99%

See 2 more Smart Citations

Deceptive learning in histopathology

Shahamatdar

Saeed-Vafa

Linsley

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning holds immense potential for histopathology, automating tasks that are simple for expert pathologists, and revealing novel biology for tasks that were previously considered difficult or impossible to solve by eye alone. However, the extent to which the visual strategies learned by deep learning models in histopathological analysis are trustworthy or not has yet to be systematically analyzed. In this work, we address this problem and discover new limits on the histopathological tasks for which deep learning models learn trustworthy versus deceptive solutions. While tasks that have been extensively studied in the field like tumor detection are reliable and trustworthy, recent advances demonstrating the ability to learn molecular profiling from hematoxylin and eosin (H&E) stained slides do not hold up to closer scrutiny. Our analysis framework represents a new approach in understanding the capabilities of deep learning models, which should be incorporated into the computational pathologists toolkit.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Task 1: Dnns Trained For Molecular Profiling Are Vulnerable ...mentioning

confidence: 99%

See 1 more Smart Citation

Deceptive learning in histopathology

Shahamatdar

Saeed-Vafa

Linsley

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep learning algorithms (LeCun et al, 2015; Yosinski et al, 2015; Serre, 2019; Goh et al, 2021) will presumably be responsive to diagnostic regions that are larger than the small sample areas used here, including traditional whole‐leaf features. Computer vision interpretability is a new and burgeoning field (Olah et al, 2018; Lapuschkin et al, 2019; Linsley et al, 2021; McGrath et al, 2021 [Preprint]; Voss et al, 2021) that, coupled with the mass digitization of herbaria and fossil plant collections, seems certain to further assist botanists and paleobotanists in the identification of both fossil and extant leaves (Beaman and Cellinese, 2012; Page et al, 2015; Hedrick et al, 2020).…”

Section: Discussionmentioning

confidence: 99%

“…Computer vision algorithms categorize complex patterns, often with a capacity far beyond humans (Gouveia et al, 1997; Linsley et al, 2021), and heat maps can be generated from experimental output to visualize diagnostic regions that were not previously noticed (Lapuschkin et al, 2019; Miao et al, 2019; McGrath et al, 2021 [Preprint]). These visualizations are important for interpreting machine‐learning results and guiding human users to discover novel information.…”

Section: Figurementioning

confidence: 99%

Decoding family‐level features for modern and fossil leaves from computer‐vision heat maps

Spagnuolo

Wilf

Serre

2022

American J of Botany

Self Cite

View full text Add to dashboard Cite

Premise Angiosperm leaves present a classic identification problem due to their morphological complexity. Computer‐vision algorithms can identify diagnostic regions in images, and heat map outputs illustrate those regions for identification, providing novel insights through visual feedback. We investigate the potential of analyzing leaf heat maps to reveal novel, human‐friendly botanical information with applications for extant‐ and fossil‐leaf identification. Methods We developed a manual scoring system for hotspot locations on published computer‐vision heat maps of cleared leaves that showed diagnostic regions for family identification. Heat maps of 3114 cleared leaves of 930 genera in 14 angiosperm families were analyzed. The top‐5 and top‐1 hotspot regions of highest diagnostic value were scored for 21 leaf locations. The resulting data were viewed using box plots and analyzed using cluster and principal component analyses. We manually identified similar features in fossil leaves to informally demonstrate potential fossil applications. Results The method successfully mapped machine strategy using standard botanical language, and distinctive patterns emerged for each family. Hotspots were concentrated on secondary veins (Salicaceae, Myrtaceae, Anacardiaceae), tooth apices (Betulaceae, Rosaceae), and on the little‐studied margins of untoothed leaves (Rubiaceae, Annonaceae, Ericaceae). Similar features drove the results from multivariate analyses. The results echo many traditional observations, while also showing that most diagnostic leaf features remain undescribed. Conclusions Machine‐derived heat maps that initially appear to be dominated by noise can be translated into human‐interpretable knowledge, highlighting paths forward for botanists and paleobotanists to discover new diagnostic botanical characters.

show abstract

Deep learning and deep phenotyping of HD iPSCs: Applications to study biology and test therapeutics

Finkbeiner

2024

Huntington's Disease

View full text Add to dashboard Cite

Superhuman cell death detection with biomarker-optimized neural networks

Abstract: High-throughput microscopy has outpaced analysis; biomarker-optimized CNNs are a generalizable, fast, and interpretable solution.

Cited by 12 publications

References 62 publications

Deceptive learning in histopathology

Deceptive learning in histopathology

Decoding family‐level features for modern and fossil leaves from computer‐vision heat maps

Deep learning and deep phenotyping of HD iPSCs: Applications to study biology and test therapeutics

Contact Info

Product

Resources

About