Modeling Visual Context Is Key to Augmenting Object Detection Datasets

Abstract

Performing data augmentation for learning deep neural networks is well known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. For object detection, classical approaches for data augmentation consist of generating images obtained by basic geometrical transformations and color changes of original training images. In this work, we go one step further and leverage segmentation annotations to increase the number of object instances present on training data. For this approach to be successful, we show that modeling appropriately the visual context surrounding objects is crucial to place them in the right environment. Otherwise, we show that the previous strategy actually hurts. With our context model, we achieve significant mean average precision improvements when few labeled examples are available on the VOC'12 benchmark.

Imbalance Problems in Object Detection: A Review

“…Alternatively, artificial images can be produced to augment the dataset [63], [64], [90], [91] by generating composite images in which multiple crops and/or images are blended. A straightforward approach is to randomly place cropped objects onto images as done by [90].…”

“…However, the produced images may look unrealistic. This problem is alleviated by determining where to paste and the size of the pasted region according to the visual context [91]. In a similar vein, the objects can be swapped between images: Progressive and Selective Instance-Switching (PSIS) [64] swaps single objects belonging to the same class between a pair of images considering also the scales and shapes of the candidate instances.…”

InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting

“…Previous method [15] investigated applying context model to explicitly model the consistency of the object and background in semantic space. Different from their approach, our appearance consistency map does not consider the semantic consistency explicitly but enforces the object to be pasted at places with similar background pattern on the original image.…”

“…Instance-level augmentation. One branch of recent work has emerged with more precise instance-level image augmentation, laying potential to fully exploit the supervised information in the existing dataset [16,26,14,15,18,25,43]. Dwibedi et al [16] improved instance detection by simple cut-and-paste strategy with extra instances that have annotated masks.…”

“…However, these data augmentation method does not work in the area of instance segmentation, as dataset priors are not efficiently exploited, resulting in poor performance in our experiments. Meanwhile, adopting a deep context model [15] introduces significant computational overhead, making it less practical in real-world applications.…”