Interleaved text/image Deep Mining on a large-scale radiology database

Shin, Hoo-Chang; Lü, Le; Kim, Lauren; Seff, Ari; Yao, Jianhua; Summers, Ronald M.

doi:10.1109/cvpr.2015.7298712

Cited by 91 publications

(100 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They showed that semantic information increases classification accuracy for a variety of pathologies in Optical Coherence Tomography (OCT) images. Shin et al (2015) and Wang et al (2016e) mined semantic interactions between radiology reports and images from a large data set extracted from a PACS system. They employed latent Dirichlet allocation (LDA), a type of stochastic model that generates a distribution over a vocabulary of topics based on words in a document.…”

Section: Combining Image Data With Reportsmentioning

confidence: 99%

A survey on deep learning in medical image analysis

Litjens

Kooi

Bejnordi

et al. 2017

Medical Image Analysis

10,138

5,414

View full text Add to dashboard Cite

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks. Concise overviews are provided of studies per application area: neuro, retinal, pulmonary, digital pathology, breast, cardiac, abdominal, musculoskeletal. We end with a summary of the current state-of-the-art, a critical discussion of open challenges and directions for future research.

show abstract

Section: Combining Image Data With Reportsmentioning

confidence: 99%

A survey on deep learning in medical image analysis

Litjens

Kooi

Bejnordi

et al. 2017

Medical Image Analysis

10,138

5,414

View full text Add to dashboard Cite

show abstract

“…26 One example for an approach based on deep learning and convolutional networks has already been published by Shin et al for X-Ray images. 27 Currently, medical retrieval systems try to become much more accessible on the web, typically being multi-modal in a way by supporting both textual and visual queries. Examples for web based search engines are NovaMedSearch 28 and GoldMiner.…”

Section: Related Workmentioning

confidence: 99%

Content-based retrieval in videos from laparoscopic surgery

et al. 2016

View full text Add to dashboard Cite

In the field of medical endoscopy more and more surgeons are changing over to record and store videos of their endoscopic procedures for long-term archival. These endoscopic videos are a good source of information for explanations to patients and follow-up operations. As the endoscope is the "eye of the surgeon", the video shows the same information the surgeon has seen during the operation, and can describe the situation inside the patient much more precisely than an operation report would do. Recorded endoscopic videos can also be used for training young surgeons and in some countries the long-term archival of video recordings from endoscopic procedures is even enforced by law. A major challenge, however, is to efficiently access these very large video archives for later purposes. One problem, for example, is to locate specific images in the videos that show important situations, which are additionally captured as static images during the procedure. This work addresses this problem and focuses on contentbased video retrieval in data from laparoscopic surgery. We propose to use feature signatures, which can appropriately and concisely describe the content of laparoscopic images, and show that by using this content descriptor with an appropriate metric, we are able to efficiently perform content-based retrieval in laparoscopic videos. In a dataset with 600 captured static images from 33 hours recordings, we are able to find the correct video segment for more than 88% of these images.Feature signatures, as for instance investigated in the work of Beecks, 1 are a compact yet efficient way of describing the content of an image. They are obtained by clustering characteristic image features into a compact feature representation that adapts to individual image contents. In this paper, we propose to utilize feature signatures over a 7-dimensional feature space including position, color, and texture information for content-based retrieval in laparoscopic videos. To this end, we investigate different distance-based similarity measures including the Earth Mover's Distance, 2 the Signature Quadratic Form Distance, 3, 4 and the Signature Matching Distance. 5 Through a large evaluation with video data from 33 hours of laparoscopic surgery we show that the proposed signature-based approaches are able to efficiently perform content-based retrieval in laparoscopic videos. Description of PurposeIn endoscopic surgery, operating surgeons usually record video segments and capture static images during the operation, which they often need after the procedure. 6 However, while the static images can be used for later discussions with patients and colleagues, and can be easily integrated into operation reports, they lack of a complete and precise information, such as movements of operation instruments and the dynamics of performed actions. The recorded videos, on the other hand, contain all these important details, since they show exactly the same images that were visible on the monitors in the operation room, which were also used by the...

show abstract

“…[7] fine-tuned all layers of a pre-trained CNN for automatic classification of interstitial lung diseases. In [21], Shin et al . used fine-tuned pre-trained CNNs to automatically map medical images to document-level topics, document-level sub-topics, and sentence-level topics.…”

Section: Related Workmentioning

confidence: 99%

“…Several researchers have demonstrated the utility of fine-tuning CNNs for biomedical image analysis, but they only performed one-time fine-tuning, that is, simply fine-tuning a pre-trained CNN once with available training samples involving no active selection processes (e.g., [4, 19, 5, 2, 21, 7, 18, 24]). To our knowledge, our proposed method is among the first to integrate active learning into fine-tuning CNNs in a continuous fashion to make CNNs more amicable for biomedical image analysis with an aim to cut annotation cost dramatically.…”

Section: Introductionmentioning

confidence: 99%

Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally

Zhou

Shin

Zhang

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

363

173

View full text Add to dashboard Cite

Intense interest in applying convolutional neural networks (CNNs) in biomedical image analysis is wide spread, but its success is impeded by the lack of large annotated datasets in biomedical imaging. Annotating biomedical images is not only tedious and time consuming, but also demanding of costly, specialty-oriented knowledge and skills, which are not easily accessible. To dramatically reduce annotation cost, this paper presents a novel method called AIFT (active, incremental fine-tuning) to naturally integrate active learning and transfer learning into a single framework. AIFT starts directly with a pre-trained CNN to seek “worthy” samples from the unannotated for annotation, and the (fine-tuned) CNN is further fine-tuned continuously by incorporating newly annotated samples in each iteration to enhance the CNN’s performance incrementally. We have evaluated our method in three different biomedical imaging applications, demonstrating that the cost of annotation can be cut by at least half. This performance is attributed to the several advantages derived from the advanced active and incremental capability of our AIFT method.

show abstract

Interleaved text/image Deep Mining on a large-scale radiology database

Cited by 91 publications

References 27 publications

A survey on deep learning in medical image analysis

A survey on deep learning in medical image analysis

Content-based retrieval in videos from laparoscopic surgery

Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally

Contact Info

Product

Resources

About