Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition

Abstract

In general image recognition problems, discriminative information often lies in local image patches. For example, most human identity information exists in the image patches containing human faces. The same situation stays in medical images as well. "Bodypart identity" of a transversal slice-which bodypart the slice comes from-is often indicated by local image information, e.g., a cardiac slice and an aorta arch slice are only differentiated by the mediastinum region. In this work, we design a multi-stage deep learning framework for image classification and apply it on bodypart recognition. Specifically, the proposed framework aims at: 1) discover the local regions that are discriminative and non-informative to the image classification problem, and 2) learn a image-level classifier based on these local regions. We achieve these two tasks by the two stages of learning scheme, respectively. In the pre-train stage, a convolutional neural network (CNN) is learned in a multi-instance learning fashion to extract the most discriminative and and non-informative local patches from the training slices. In the boosting stage, the pre-learned CNN is further boosted by these local patches for image classification. The CNN learned by exploiting the discriminative local appearances becomes more accurate than those learned from global image context. The key hallmark of our method is that it automatically discovers the discriminative and non-informative local patches through multi-instance deep learning. Thus, no manual annotation is required. Our method is validated on a synthetic dataset and a large scale CT dataset. It achieves better performances than state-of-the-art approaches, including the standard deep CNN.

Landmark-based deep multi-instance learning for brain disease diagnosis

“…Recently, MIL has been adopted in the medical imaging analysis domain (Bi and Liang, 2007; Liu et al, 2010; Lu et al, 2011; Xu et al, 2012; Tong et al, 2014; Xu et al, 2014; Yan et al, 2016). In Lu et al (2011) and Xu et al (2012), MIL-like methods were developed to perform medical image segmentation.…”

“…This method adopted intensity values within a patch for feature representation that was independent of the subsequent SVM classifier. More recently, a multi-instance deep learning method (Yan et al, 2016) was developed to discover discriminative local anatomies for body-part recognition. This method consisted of a two-stage CNN model, where the first-stage CNN was trained in a multi-instance learning fashion to locate discriminative image patches, and the second-stage CNN was boosted using those selected patches.…”

Learning laparoscopic video shot classification for gynecological surgery

“…Their results imply that, for this use case, their CNN approach outperforms other CNNs as well as state-of-the-art methods using handcrafted features. In the work of Yan et al [29], a multi-stage deep learning framework is presented. Using the proposed framework, the authors try to solve the problem of body-part recognition in MRI images.…”

Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique

“…Yan et al [33] design a multi-stage deep learning framework for image classification and apply it on body part recognition. In the pre-train stage, a convolutional neural network (CNN) is learned using multi-instance learning to extract the most discriminative and non-informative local patches from the training slices.…”