Adapting Object Detectors from Images to Weakly Labeled Videos

Chanda, Omit; Teh, Eu Wern; Rochan, Mrigank; Guo, Zhenyu; Wang, Yang

doi:10.5244/c.31.56

Cited by 4 publications

(4 citation statements)

References 15 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-domain object detection. The domain shift [29] of detectors trained on still images and applied to video frames has been addressed in several works, mostly relying on some form of weak supervision on the target domain and selecting target samples based on the baseline detector confidence score [19,54,47,10,30,6]. Several approaches have used weakly-labeled video data for re-training object detectors [27,49,54].…”

Section: Related Workmentioning

confidence: 99%

Automatic Adaptation of Object Detectors to New Domains Using Self-Training

RoyChowdhury

Chakrabarty

Singh

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

157

View full text Add to dashboard Cite

This work addresses the unsupervised adaptation of an existing object detector to a new target domain. We assume that a large number of unlabeled videos from this domain are readily available. We automatically obtain labels on the target data by using high-confidence detections from the existing detector, augmented with hard (misclassified) examples acquired by exploiting temporal cues using a tracker. These automatically-obtained labels are then used for re-training the original model. A modified knowledge distillation loss is proposed, and we investigate several ways of assigning soft-labels to the training examples from the target domain. Our approach is empirically evaluated on challenging face and pedestrian detection tasks: a face detector trained on WIDER-Face, which consists of highquality images crawled from the web, is adapted to a largescale surveillance data set; a pedestrian detector trained on clear, daytime images from the BDD-100K driving data set is adapted to all other scenarios such as rainy, foggy, nighttime. Our results demonstrate the usefulness of incorporating hard examples obtained from tracking, the advantage of using soft-labels via distillation loss versus hard-labels, and show promising performance as a simple method for unsupervised domain adaptation of object detectors, with minimal dependence on hyper-parameters. Code and models are available at

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic Adaptation of Object Detectors to New Domains Using Self-Training

RoyChowdhury

Chakrabarty

Singh

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

157

View full text Add to dashboard Cite

show abstract

“…To circumnavigate this requirement there are two genres of approach closely related to our current effort. One line of approach is weakly supervised object localization [1,17,23,24,30,3], wherein we only have meta information such as the presence/absence of an object category. Majority of these algorithms are based on multiple instance learning(MIL) framework.…”

Section: Object Detectionmentioning

confidence: 99%

“…During inference, transformation is done on 256×256 resized image and then rescaled to the original resolution before object detection. We have modelled the generator with 9 resnet blocks as implemented in [13] and the discriminator with PatchGAN classifier [14,19] 3 . GAN loss is implemented with vanilla GAN [12] objective, while cycle loss is L 1 loss.…”

Section: Cycleganmentioning

confidence: 99%

“…In [24] (proposal only) refers to learning of appearance model based on object proposals on weakly annotated frames while (proposal + transfer) refers to combination of appearance model from 'novel' objects (video) and transferred appearance model from 'familiar' annotated objects (static images). Method of Chanda et al [3] is based on a 2-stream network, wherein one stream they perform regular fully supervised image object detection and in another stream, they perform frame level classification on the weakly annotated videos. These 2 streams share parameters to counter domain shift factors.…”

Section: Comparing Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Adversarial Visual Level Domain Adaptation for Learning Video Object Detectors From Images

Lahiri

Ragireddy²,

Biswas

et al. 2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Deep learning based object detectors require thousands of diversified bounding box and class annotated examples. Though image object detectors have shown rapid progress in recent years with release of multiple large scale static image datasets, object detection on videos still remains an open problem due to scarcity of annotated video frames. Having a robust video object detector is an essential component for video understanding and curating large scale automated annotations in videos. Domain difference between images and videos makes the transferability of image object detectors to videos sub-optimal. The most common solution is to use weakly supervised annotations where a video frame has to be tagged for presence/absence of object categories. This still takes up manual effort. In this paper we take a step forward by adapting the concept of unsupervised adversarial image-to-image translation to perturb static high quality images to be visually indistinguishable from a set of video frames. We assume the presence of a fully annotated static image dataset and an unannotated video dataset. Object detector is trained on adversarially transformed image dataset using the annotations of the original dataset. Experiments on Youtube-Objects and Youtube-Objects-Subset datasets with two contemporary baseline object detectors reveal that such unsupervised pixel level domain adaptation boosts the generalization performance on video frames compared to direct application of original image object detector. Also, we achieve competitive performance compared to recent baselines of weakly supervised methods. This paper can be seen as an application of image translation for cross domain object detection. * Equal contribution Figure 1: Exemplary success of our model in detecting objects in high speed and blurry video frames. Left: Detection by Faster R-CNN [9] object detector trained on static images of PASCAL VOC. Right: Detection by the same Faster R-CNN framework but trained on our proposed adversarially transformed PASCAL VOC images. Our video class agnostic framework achieves near to state-of-the-art results compared to weakly supervised methods.

show abstract