Towards Accurate One-Stage Object Detection With AP-Loss

Abstract

One-stage object detectors are trained by optimizing classification-loss and localization-loss simultaneously, with the former suffering much from extreme foregroundbackground class imbalance issue due to the large number of anchors. This paper alleviates this issue by proposing a novel framework to replace the classification task in one-stage detectors with a ranking task, and adopting the Average-Precision loss (AP-loss) for the ranking problem. Due to its non-differentiability and non-convexity, the APloss cannot be optimized directly. For this purpose, we develop a novel optimization algorithm, which seamlessly combines the error-driven update scheme in perceptron learning and backpropagation algorithm in deep networks. We verify good convergence property of the proposed algorithm theoretically and empirically. Experimental results demonstrate notable performance improvement in state-ofthe-art one-stage detectors based on AP-loss over different kinds of classification-losses on various benchmarks, without changing the network architectures. Code is available at https://github.com/cccorn/AP-loss.

Imbalance Problems in Object Detection: A Review

“…regression). To illustrate, AP Loss [60] does not modify the regression branch; however, COCO style AP @0.75 increases around 3%. This example shows that the loss functions for different branches (tasks) are not independent (see also Figure 15).…”

“…Explanation. Chen et al [60] use all the confidence scores all together without paying attention to this. That's why, there are two possible results: (i) Either the AP Loss is robust to the variations in the meanings of the confidence scores of different classes which can be due to the ranking task it uses, or (ii) if not, then a method sorting the confidence scores in a class specific manner and then combines them to generate the final AP Loss is expected to perform better, which remains as an open problem.…”

AP-Loss for Accurate One-Stage Object Detection

“…This manuscript is an extension of our CVPR 2019 paper [36], we make the following contributions. We address the imbalance issue by replacing the classification task in one-stage detectors with a ranking task, so that the class imbalance problem can be handled with Average Precision (AP)-loss, a rank-based loss.…”

Pedestrian as Points: An Improved Anchor-Free Method for Center-Based Pedestrian Detection

“…Illumination-aware faster R-CNN [50] addresses the problem of fusing color and thermal modalities for detecting multispectral images. In the one-stage stream, a considerable number of approaches [51][52][53][54][55] which use anchor mechanism are proposed after SSD [25]. They aim at improving performance, including multi-stage refinement [51,52], adaptive anchors [53] and loss function improvement [54,55].…”

“…The predicted heatmaps are with the same size as the concatenated feature maps. Note that more complicated detection head like [52,55] can be explored to further improve the detection performance, but it beyond the scope of this work.…”