Unsupervised Joint Mining of Deep Features and Image Labels for Large-Scale Radiology Image Categorization and Scene Recognition

Abstract

The recent rapid and tremendous success of deep convolutional neural networks (CNN) on many challenging computer vision tasks largely derives from the accessibility of the well-annotated ImageNet and PASCAL VOC datasets. Nevertheless, unsupervised image categorization (i.e., without the ground-truth labeling) is much less investigated, yet critically important and difficult when annotations are extremely hard to obtain in the conventional way of "Google Search" and crowd sourcing. We address this problem by presenting a looped deep pseudo-task optimization (LDPO) framework for joint mining of deep CNN features and image labels. Our method is conceptually simple and rests upon the hypothesized "convergence" of better labels leading to better trained CNN models which in turn feed more discriminative image representations to facilitate more meaningful clusters/labels. Our proposed method is validated in tackling two important applications: 1) Large-scale medical image annotation has always been a prohibitively expensive and easily-biased task even for well-trained radiologists. Significantly better image categorization results are achieved via our proposed approach compared to the previous state-of-the-art method. 2) Unsupervised scene recognition on representative and publicly available datasets with our proposed technique is examined. The LDPO achieves excellent quantitative scene classification results. On the MIT indoor scene dataset, it attains a clustering accuracy of 75.3%, compared to the state-of-the-art supervised classification accuracy of 81.0% (when both are based on the VGG-VD model) .

CNN Features are also Great at Unsupervised Classification

“…Regarding the results of this methodology on Scene clustering [18], it appears that fine tuning feature extraction using alternating optimization is a good way of improving clustering results. However, the simple approach proposed here still keeps the advantage of being very fast (as it only requires to evaluate the network once for each sample and apply AC), which is useful for our application for instance.…”

“…It has applications for searching large image database [13,14,15], concept discovery in images [16], storyline reconstruction [17], medical images classification [18], ... The first successful methods focused on feature selection and used sophisticated algorithms to deal with complex features.…”

“…[10] uses deep auto-encoders combined with ensemble clustering to generate feature representations suitable for clustering. [22,18] learns jointly the clusters and the representation using alternating optimization [23].…”

“…For this reason, we compared our result at different object categorization tasks as well as fine-grained classification for clustering of similar objects. Another interesting image classification problem is the one of scene clustering, studied in [18] on two datasets ( [42,43]). For this task, the pipeline proposed in this paper cannot perform as well as for object classification.…”

Visually Supporting Image Annotation Based on Visual Features and Ontologies

“…In this context, several image annotation and retrieval methods based on various learning techniques have been applied [17] and [8]. Moreover, image annotation could be presented as a classification problem when the goal is to improve image classification and annotation accuracy [9], [18] and [19]. Several works have focused on scene recognition and image classification using features learning with Convolutional Neural Networks (CNNs) [20], [5] and [12].…”