Bag‐of‐binary‐features for fast image representation

Abstract

The possibility of integrating binary features into the bag-of-features (BoFs) model is explored. The set of binary features extracted from an image are packed into a single vector form, to yield the bag-ofbinary-features (BoBFs). The efficient BoBF feature extraction and quantisation provide fast image representation. The trade-off between accuracy and efficiency in BoBF compared with BoF is investigated through image retrieval tasks. Experimental results demonstrate that BoBF is a competitive alternative to BoF when the run-time efficiency is critical.Introduction: Image representation is a fundamental issue in many computer vision tasks, such as scene classification, image categorisation and image retrieval. These tasks are challenging on account of significant object appearance variabilities caused by non-rigidity, occlusion, background cluttering and pose and lighting changes that occur in realworld images. To address these variabilities, local image appearances are modelled using local features. The local features are very robust to the variabilities by capturing the invariant aspects of local regions.Inspired by a text-retrieval approach, bag-of-features (BoFs) [1] integrate these local features into a single vector representation. In BoF-based image representation, the set of local features extracted from an image is quantised or clustered into a vocabulary of visual words. An image is then represented by a vector of visual word frequencies obtained by indexing each feature of the image to the closest visual word. By leveraging local features, BoF-based image representation has demonstrated excellent performance for various computer vision tasks.In general, BoF builds on gradient-based local features, such as the scale-invariant feature transform (SIFT) descriptor [2]. Extracting these features is a time-intensive task, which results in a computational burden on BoF. As reported in [3], in BoF-based image representation, the majority of the computational cost is from extracting local features. This makes BoF unsuitable for some real-world applications, particularly for embedded applications in which run-time efficiency is critical. Meanwhile, efficient alternative local features, called binary features, have been recently studied. Comprised of a sequence of bit patterns, binary features are efficiently extracted by simple intensity comparisons.In this Letter, we present the bag-of-binary-features (BoBFs) by integrating the binary features into BoF. The fast feature extraction and quantisation of BoBF make it much faster than BoF in image representation. We investigate the trade-off between accuracy and efficiency of BoBF compared with BoF through image retrieval tasks. Our experimental results demonstrate that BoBF is a competitive alternative to BoF for fast image representation.

Quantized Deep Residual Convolutional Neural Network for Image-Based Dietary Assessment

“…It is common for BoF to use SIFT as visual detectors for feature extractions. However, SIFT is a very time-consuming task and often failed to extract sufficient interest points for classification problems [27]. Anthimopoulos et al [28] conducted multiple experiments to find out an optimized BoF model for food recognition.…”

Aggregating binary local descriptors for image retrieval

“…In [21] the visual vocabulary was calculated by binarizing the centroids obtained using the standard k-means. In [82,26,45] the k-means clustering was modified to fit the binary features by replacing the Euclidean distance with the Hamming distance, and by replacing the mean operation with the median operation. In [76] the VLAD image signature was adapted to work with binary descriptors: kmeans is used for learning the visual vocabulary and the VLAD vectors are computed in conjunction with an intra-normalization and a final binarization step.…”

“…Since quantization and aggregation methods are defined and used almost exclusively in conjunction with non-binary features, the cost of extracting local descriptors and to quantize/aggregate them on the fly, is still high. Recently, some approaches that attempt to integrate the binary local descriptors with the quantization and aggregation methods have been proposed in literature [21,26,45,76,73,82]. In these proposals, the aggregation is directly applied on top of binary local descriptors.…”

Quantisation‐free binary feature aggregation for fast image representation

“…As compared to the BoF technique, the proposed method achieved a considerable gain in computational speed with only marginal loss in accuracy. As reported in [4], this was due to the trade-off between accuracy and computational efficiency of the binary feature itself. In addition, the proposed method unexpectedly outperformed the BoBF method, which implemented the same binary feature, both in accuracy and computational efficiency.…”

“…These operations are very computationally cheap, and it has been shown that extraction of binary features is nearly two orders of magnitude faster than that of gradient‐based local features. In [4], to achieve fast image representation, binary features are aggregated into a single vector by implementing a BoF algorithm to yield bag‐of‐binary features (BoBFs). Since binary features consist of a sequence of bit strings, to aggregate these binary features, k ‐means clustering has been modified to fit the binary features by implementing Hamming distance and median operation instead of typical Euclidean distance and mean operation, respectively.…”