Deep Learning: A Primer for Radiologists

Chartrand, Gabriel; Cheng, Phillip M.; Vorontsov, Eugene; Drozdzal, Michal; Turcotte, Simon; Pal, Christopher; Kadoury, Samuel; Tang, An

doi:10.1148/rg.2017170077

Cited by 924 publications

(687 citation statements)

References 35 publications

Supporting

Mentioning

667

Contrasting

Unclassified

Order By: Relevance

“…The encoder-decoder networks are widely used in modern semantic and instance segmentation models [1], [2], [3], [4], [5], [6]. Their success is largely attributed to their skip connections, which combine deep, semantic, coarse-grained feature maps from the decoder sub-network with shallow, lowlevel, fine-grained feature maps from the encoder sub-network, and have proven to be effective in recovering fine-grained details of the target objects [7], [8], [9] even on complex background [10], [11].…”

Section: Introductionmentioning

confidence: 99%

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

Zhou

Siddiquee

Tajbakhsh

et al. 2020

IEEE Trans. Med. Imaging

2,586

1,038

View full text Add to dashboard Cite

The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations:(1) their optimal depth is apriori unknown, requiring extensive architecture search or inefficient ensemble of models of varying depths; and (2) their skip connections impose an unnecessarily restrictive fusion scheme, forcing aggregation only at the samescale feature maps of the encoder and decoder sub-networks. To overcome these two limitations, we propose UNet++, a new neural architecture for semantic and instance segmentation, by (1) alleviating the unknown network depth with an efficient ensemble of U-Nets of varying depths, which partially share an encoder and co-learn simultaneously using deep supervision; (2) redesigning skip connections to aggregate features of varying semantic scales at the decoder sub-networks, leading to a highly flexible feature fusion scheme; and (3) devising a pruning scheme to accelerate the inference speed of UNet++. We have evaluated UNet++ using six different medical image segmentation datasets, covering multiple imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and electron microscopy (EM), and demonstrating that (1) UNet++ consistently outperforms the baseline models for the task of semantic segmentation across different datasets and backbone architectures; (2) UNet++ enhances segmentation quality of varying-size objects-an improvement over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design) outperforms the original Mask R-CNN for the task of instance segmentation; and (4) pruned UNet++ models achieve significant speedup while showing only modest performance degradation. Our implementation and pre-trained models are available at https://github.com/MrGiovanni/UNetPlusPlus.

show abstract

Section: Introductionmentioning

confidence: 99%

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

Zhou

Siddiquee

Tajbakhsh

et al. 2020

IEEE Trans. Med. Imaging

2,586

1,038

View full text Add to dashboard Cite

show abstract

“…It has been observed that large-scaled datasets for general purpose have been annotated though crowdsourcing (i.e., outsourcing of the annotation tasks to a variety of competent sources) to 40 generate ground-truth labeled ones. In healthcare, to date, though crowdsourcing has shown high potentials, such as optimizing treatment plans, predicting disease outbreaks, patient centered price transparency (Meisel et al, 2016) and drug discovery (Lessl et al, 2011), few work is initiated to annotate the medical images by crowdsourcing since outsourcing of the medical annotation work to non-expertise would lead to misclassifications (Chartrand et al, 2017;Weese and Lorenz, 2016;Zygmont et al, 2016). Some steps in this direction could be referred to the work by Albarqouni et al (Albarqouni et al, 2016).…”

Section: Crowdsourcing To Build Ground-truth Labeled Datasetmentioning

confidence: 99%

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning-aided decision support: three decades’ development course and future prospect

Liu

Chi

et al. 2019

J Cancer Res Clin Oncol

View full text Add to dashboard Cite

Highlights► It covers the literature of the past thirty years' development in computer-assisted diagnosis of lung nodules.► A summary of algorithms from classical approaches to deep learning on pulmonary nodules diagnosis is provided.► Challenges and opportunities in learning models, algorithms and schemes are highlighted. AbstractLung cancer is the commonest cause of cancer deaths worldwide, and its mortality can be reduced significantly by performing early diagnosis and screening. Since the 1960s, driven by the pressing needs to accurately and effectively interpret the massive volume of chest images generated daily, computer-assisted diagnosis of pulmonary nodule has opened up new opportunities to relax the limitation from physicians' subjectivity, experiences and fatigue. And the fair access to the reliable and affordable computer-assisted diagnosis will fight the inequalities in incidence and mortality between populations. It has been witnessed that significant and remarkable advances have been achieved since the 1980s, and consistent endeavors have been exerted to deal with the grand challenges on how to accurately detect the pulmonary nodules with high sensitivity at low false-positives rate as well as on how to precisely differentiate between benign and malignant nodules. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the computer-assisted nodules detection and benign-malignant classification techniques developed over three decades, which have evolved from the complicated ad hoc analysis pipeline of conventional approaches to the simplified seamlessly integrated deep learning techniques. This review also identifies challenges and highlights opportunities for future work in learning models, learning algorithms and enhancement schemes for bridging current state to future prospect and satisfying future demand. As far as the authors know, it is the first review of the literature of the past thirty years' development in computer-assisted diagnosis of lung nodules. We acknowledge the value of potential multidisciplinary researches that will make the computer-assisted diagnosis of pulmonary nodules enter into the main stream of clinical medicine, and raise the state-of-the-art clinical applications as well as increase both welfares of physicians and patients.

show abstract

“…At the same time, the groundbreaking performance of convolutional neural networks in computer vision such as ImageNet has led to a rapid ascent of interest in the application of deep learning to medical imaging . In cases where the sample size is small, deep learning can be used for representation learning.…”

Section: Introductionmentioning

confidence: 99%

“…[13][14][15] At the same time, the groundbreaking performance of convolutional neural networks in computer vision such as ImageNet 16 has led to a rapid ascent of interest in the application of deep learning to medical imaging. 17,18 In cases where the sample size is small, deep learning can be used for representation learning. The key idea is to use the CNN to infer suitable feature representation for an objective task (e.g., classification) rather than to perform an end-to-end classification.…”

Section: Introductionmentioning

confidence: 99%

Deep learning for patient‐specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks

et al. 2018

View full text Add to dashboard Cite

Purpose Patient‐specific quality assurance (QA) for intensity‐modulated radiation therapy (IMRT) is a ubiquitous clinical procedure, but conventional methods have often been criticized as being insensitive to errors or less effective than other common physics checks. Recently, there has been interest in the application of radiomics, quantitative extraction of image features, to radiotherapy QA. In this work, we investigate a deep learning approach to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient‐specific QA. Methods Planar dose maps from 186 IMRT beams from 23 IMRT plans were evaluated. Each plan was transferred to a cylindrical phantom CT geometry. Three sets of planar doses were exported from each plan corresponding to (a) the error‐free case, (b) a random multileaf collimator (MLC) error case, and (c) a systematic MLC error case. Each plan was delivered to the electronic portal imaging device (EPID), and planned and measured doses were used to calculate gamma images in an EPID dosimetry software package (for a total of 558 gamma images). Two radiomic approaches were used. In the first, a convolutional neural network with triplet learning was used to extract image features from the gamma images. In the second, a handcrafted approach using texture features was used. The resulting metrics from both approaches were input into four machine learning classifiers (support vector machines, multilayer perceptrons, decision trees, and k‐nearest‐neighbors) in order to determine whether images contained the introduced errors. Two experiments were considered: the two‐class experiment classified images as error‐free or containing any MLC error, and the three‐class experiment classified images as error‐free, containing a random MLC error, or containing a systematic MLC error. Additionally, threshold‐based passing criteria were calculated for comparison. Results In total, 303 gamma images were used for model training and 255 images were used for model testing. The highest classification accuracy was achieved with the deep learning approach, with a maximum accuracy of 77.3% in the two‐class experiment and 64.3% in the three‐class experiment. The performance of the handcrafted approach with texture features was lower, with a maximum accuracy of 66.3% in the two‐class experiment and 53.7% in the three‐class experiment. Variability between the results of the four machine learning classifiers was lower for the deep learning approach vs the texture feature approach. Both radiomic approaches were superior to threshold‐based passing criteria. Conclusions Deep learning with convolutional neural networks can be used to classify the presence or absence of introduced radiotherapy treatment delivery errors from patient‐specific gamma images. The performance of the deep learning network was superior to a handcrafted approach with texture features, and both radiomic approaches were better than threshold‐based passing criteria. The results suggest that radiomic QA is a promising direction f...

show abstract

Deep Learning: A Primer for Radiologists

Cited by 924 publications

References 35 publications

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning-aided decision support: three decades’ development course and future prospect

Deep learning for patient‐specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks

Contact Info

Product

Resources

About