Compact Bilinear Pooling

Abstract

Bilinear models has been shown to achieve impressive performance on a wide range of visual tasks, such as semantic segmentation, fine grained recognition and face recognition. However, bilinear features are high dimensional, typically on the order of hundreds of thousands to a few million, which makes them impractical for subsequent analysis. We propose two compact bilinear representations with the same discriminative power as the full bilinear representation but with only a few thousand dimensions. Our compact representations allow back-propagation of classification errors enabling an end-to-end optimization of the visual recognition system. The compact bilinear representations are derived through a novel kernelized analysis of bilinear pooling which provide insights into the discriminative power of bilinear pooling, and a platform for further research in compact pooling methods. Experimentation illustrate the utility of the proposed representations for image classification and few-shot learning across several datasets.

WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation

“…VOC 2007 VOC 2012 VGG16 [56] 89.3 89.0 DeepMIL [44] -86.3 WELDON [13] 90.2 -ResNet-101 (*) [28] 89.8 89.2 ProNet [58] -89.3 RRSVM [61] 92.9 -SPLeaP [35] 88.0 -WILDCAT 95.0 93.4 In Table 3, we compare WILDCAT results for scene categorization with recent global image representations used for image classification: deep features [71,28], and global image representation with deep features computed on image regions: MOP CNN [25] and Compact Bilinear Pooling [18]. Again, WILDCAT gets the best results, showing the capacity of our model to seek discriminative part regions, whereas background and non-informative parts are incorporated into image representation with other approaches.…”

“…15 Scene MIT67 CaffeNet Places [71] 90.2 68.2 MOP CNN [25] -68.9 Negative parts [47] -77.1 GAP GoogLeNet [70] 88.3 66.6 WELDON [13] 94.3 78.0 Compact Bilinear Pooling [18] -76.2 ResNet-101 (*) [28] 91.9 78.0 SPLeaP [35] -73.5 WILDCAT 94.4 84.0 Table 3. Classification performances (multi-class accuracy) on scene datasets.…”

Deep bilinear features for Her2 scoring in digital pathology

“…Therefore, the Her2 scoring class is related to the area of fine-grained recognition [5,6,8] as well as texture classification [7]. The recently successful concept of bilinear features [1,5] of this area is used in this paper. We randomly 1 crop several 227 x 227 patches at resolution level 1 (highest resolution is at level 0) within a 1024 x 1024 window around the click location.…”

“…As far as we know, they have not been used for biomedical applications before. In our current approach, we use signed 1 We used a fixed random seed to ensure reproducibility.…”

Robust sound event classification with bilinear multi-column ELM-AE and two-stage ensemble learning

“…This bilinear outer product can capture pairwise correlations between the feature channels to help improve feature representation. While from the perspective of kernel functions, bilinear features are closely similar to the quadratic kernel, which gives a linear classifier the discriminative power of a quadratic kernel machine [30]. Therefore, the bilinear model can be effectively used for feature representation [29,30], which is also feasible for MC-ELM-AE to build a bilinear model-based MC-ELM-AE (B-MC-ELM-AE); that is to say, the bilinear model can be applied to each pair of ELM-AE columns to further improve the robust feature representation.…”