Improving the Fisher Kernel for Large-Scale Image Classification

Abstract

Abstract. The Fisher kernel (FK) is a generic framework which combines the benefits of generative and discriminative approaches. In the context of image classification the FK was shown to extend the popular bag-of-visual-words (BOV) by going beyond count statistics. However, in practice, this enriched representation has not yet shown its superiority over the BOV. In the first part we show that with several well-motivated modifications over the original framework we can boost the accuracy of the FK. On PASCAL VOC 2007 we increase the Average Precision (AP) from 47.9% to 58.3%. Similarly, we demonstrate state-of-the-art accuracy on CalTech 256. A major advantage is that these results are obtained using only SIFT descriptors and costless linear classifiers. Equipped with this representation, we can now explore image classification on a larger scale. In the second part, as an application, we compare two abundant resources of labeled images to learn classifiers: ImageNet and Flickr groups. In an evaluation involving hundreds of thousands of training images we show that classifiers learned on Flickr groups perform surprisingly well (although they were not intended for this purpose) and that they can complement classifiers learned on more carefully annotated datasets.

Classifier Combination Using Random Walks on the Space of Concepts

“…The image representation we used was the improved Fisher Vector (FV) but without spatial pyramids (cf. [12]). Before going into details regarding the experimental procedure, we give a brief overview of this state-of-the-art image signature.…”

“…Before going into details regarding the experimental procedure, we give a brief overview of this state-of-the-art image signature. Details can be found in [11,12].…”

HEp-2 staining pattern recognition at cell and specimen levels: Datasets, algorithms and results

“…Then LOAD feature is calculated on round patches densely extracted from the eigth images (the original plus the seven in different scales). The Improved Fisher Vector (IFV) by Perronnin et al (2010) is used to encode the features to preserve its discriminative power. Then, classification is performed through a linear SVM.…”

Label embedding for text recognition

“…These patch statistics are then aggregated at an image level. We choose to compute the patch statistics using the Fisher Vector (FV) principle [19], since it obtained state-of-the-art results in image retrieval [5] and classification [2]. We assume that we have a generative model of patches (a Gaussian Mixture Model in our case) and measure the gradient of the log-likelihood of the descriptor with respect to the model parameters.…”

“…To include spatial information about the word image into the signature, we can partition the image into regions, aggregate the per-patch statistics at a region level and then concatenate the region-level signatures as proposed for instance in [8]. See [19] for more details about the FV.…”