Segmentation Driven Object Detection with Fisher Vectors

Abstract

We present an object detection system based on the Fisher vector (FV) image representation computed over SIFT and color descriptors. For computational and storage efficiency, we use a recent segmentation-based method to generate class-independent object detection hypotheses, in combination with data compression techniques. Our main contribution is a method to produce tentative object segmentation masks to suppress background clutter in the features. Re-weighting the local image features based on these masks is shown to improve object detection significantly. We also exploit contextual features in the form of a full-image FV descriptor, and an inter-category rescoring mechanism. Our experiments on the PASCAL VOC 2007 and 2010 datasets show that our detector improves over the current state-of-the-art detection results.

Bags of Spacetime Energies for Dynamic Scene Recognition

“…Here, spatiotemporal oriented features serve as primitives, owing to their strong performance in previous evaluations of dynamic scene recognition features. Recently, the Fisher vector representation [19] has shown state-of-the-art results for a variety of visual tasks (e.g., [5,14,18,23]); in contrast, here it is found that locality constrained linear coding [27] performs particularly well for dynamic scenes.…”

“…As in the original publications, pooling is performed by taking the average (VQ) or maximum (LLC) of the encoded features. For the proposed dynamic pooling, let V w and V h denote the width and height of the spacetime volume in the filtering process (5), then the integration region is set to R x = Vw 4 , R y = V h 4 , and R t is set to the temporal support of the largest filter used in (5). A 25 × K = 5000 dimensional feature vector is generated by the dynamic spacetime pyramid that also uses a hierarchical 3-level pyramid with the finest grid size of 4 × 4 for embedding geometry in 20 of the 25 channels.…”

Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

“…Analogously, traditional supervised methods for learning models of object classes from still images (Cootes et al 1998;Felzenszwalb and Huttenlocher 2005;Bourdev and Malik 2009;Felzenszwalb et al 2010;Girshick et al 2014) do not easily transfer to videos as they require expensive location annotations. The alignments recovered by our method could potentially replace the manual correspondences needed by most popular methods for learning object classes (Dalal and Triggs 2005;Felzenszwalb et al 2010;Viola et al 2005;Cinbis et al 2013;Girshick et al 2014), including those requiring part-level annotations (Felzenszwalb and Huttenlocher 2005;Bourdev and Malik 2009;Azizpour and Laptev 2012). They can also enable annotating large collections with little manual effort via knowledge transfer (Vezhnevets and Ferrari 2014;Kuettel et al 2012;Lampert et al 2009;Fei-Fei et al 2007;Malisiewicz et al 2011).…”

Patch-Level Spatial Layout for Classification and Weakly Supervised Localization

“…Best results are achieved with strongly supervised training [5,9,15,23] where object locations have to be annotated with bounding boxes. However, the annotation process is difficult, time-consuming and errorprone, especially when the objects are small.…”