Co-Occurrence Matrix Analysis-Based Semi-Supervised Training for Object Detection

Abstract

One of the most important factors in training object recognition networks using convolutional neural networks (CNNs) is the provision of annotated data accompanying human judgment. Particularly, in object detection or semantic segmentation, the annotation process requires considerable human effort. In this paper, we propose a semi-supervised learning (SSL)-based training methodology for object detection, which makes use of automatic labeling of un-annotated data by applying a network previously trained from an annotated dataset. Because an inferred label by the trained network is dependent on the learned parameters, it is often meaningless for re-training the network. To transfer a valuable inferred label to the unlabeled data, we propose a re-alignment method based on co-occurrence matrix analysis that takes into account one-hot-vector encoding of the estimated label and the correlation between the objects in the image. We used an MS-COCO detection dataset to verify the performance of the proposed SSL method and deformable neural networks (D-ConvNets) [1] as an object detector for basic training. The performance of the existing state-of-the-art detectors YOLO v2 [2], and single shot multi-box detector (SSD) [3]) can be improved by the proposed SSL method without using the additional model parameter or modifying the network architecture.

Training object detectors from few weakly-labeled and many unlabeled images

“…These are often called semisupervised [50,57]. However, semi-supervised may also refer to the situation where some images are labeled (at image-level or with bounding boxes) and the rest have no annotation at all [6,31]. This situation is consistent with the standard definition of semi-supervised learning [5].…”

“…This includes pseudo-label [23], where classifier predictions on unlabeled data are used as labels along with true labels on labeled data. The few exceptions focusing on detection [6,31] still assume there are enough labeled images to learn an object detector in the first place, which is not the case in our work. Dong et al [7] use few images with object bounding boxes and class labels along with many unlabeled images.…”

“…In object detection, this would normally translate to a small number of images having labels and bounding boxes, and a large number of images having no annotation at all. This problem has been studied for the case where the fully annotated data (with bounding boxes) are enough to train a detector in the first place [6,31], resulting in twostage learning. But what if these data are very scarce?…”

Semi-Supervised Object Detection With Sparsely Annotated Dataset

“…Semi-supervised learning for object detection. Object detection using semi-supervised learning is used in situations where it is difficult to manually acquire a sufficient number of annotations to learn, or when pseudo labels are to be obtained from a relatively large number of unlabeled data [16,17,18,19]. In [16], the author's proposed an iterative framework for evaluating and retraining pseudo-labels using pre-trained object detectors and robust trackers to obtain good pseudo-labels in successive frames.…”

“…In [17], it was possible to achieve improved detection performance in the Open Image Dataset V4 by utilizing part-aware sampling and RoI proposals to obtain good pseudo labels for sparsely annotated large-scale datasets. In [18], to efficiently use unlabeled data from the MS-COCO dataset, co-current matrix analysis was used to generate good pseudo labels by using prior information of the labeled dataset. The proposed single-object tracker-based semi-supervised learning is similar to [16] in that it uses a tracker, but has a difference in obtaining dense annotation information for a specific image by using the existing lean annotation information.…”