A Novel Rate Control Framework for SIFT/SURF Feature Preservation in H.264/AVC Video Compression

Abstract

This paper presents a novel rate control framework for H.264/Advanced Video Coding-based video coding that improves the preservation of gradient-based features like scale-invariant feature transform or speeded up robust feature compared with the default rate control algorithm in the JM reference software. First, a criterion (matching score) for feature preservation on the basis of the bag-of-features concept is proposed. Then, the matching scores are collected as a function of the quantization parameters and analyzed for different feature types. With this analysis, macroblocks are categorized into different groups before encoding. Our rate control algorithm assigns different quantization parameters to each group according to the importance of the group for feature extraction. The experimental results show that our rate control algorithm achieves the desired target bit rate, and more features are preserved compared with videos encoded using the default rate control. The proposed approach not only improves feature preservation, but also leads to a noticeable performance improvement in a real image retrieval system. The rate control framework proposed in this paper is fully standard compatible.Index Terms-Bag-of-features (BoFs), evaluation, H.264/Advanced Video Coding (AVC), rate control, scale-invariant feature transform (SIFT), speeded up robust feature (SURF).

Keypoint encoding and transmission for improved feature extraction from compressed images

“…The compressed images are transmitted via a wireless network to a server where features are extracted. In our previous work [14], we have observed that for feature-preserving image/video compression, the keypoint detection is easily affected by compression artifacts, while the descriptor calculation is more robust. To address this issue, we investigate in this paper whether explicit signaling of keypoint information (location, scale, and orientation) improves the quality of features which are extracted from the compressed images.…”

SEPIM: Secure and Efficient Private Image Matching

“…SURF approximates or even outperforms a previously proposed SIFT algorithm [28,29], which is patented, with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. Thus, the method posited here improves feature extraction speed.…”

“…Most feature vectors are either global vectors, such as a global color histogram, or local vectors such as SIFT descriptors [28,29] and SURF descriptors [6,7]. The first model generates an extreme compressed feature vector for each image.…”

High Efficiency Compression for Object Detection

“…The motivation is that gradients are useful features in a number of computer vision problems, so well-preserved gradients will likely improve the accuracy of the vision pipeline. Another recent work [6] develops a rate control scheme for H.264/AVC video coding that preserves SIFT [7] and SURF [8] features, which have also been found useful in many computer vision problems. These studies ( [5,6]) have proposed ways to preserve well-known handcrafted features through the compression process, without focusing on any particular problem.…”

“…Another recent work [6] develops a rate control scheme for H.264/AVC video coding that preserves SIFT [7] and SURF [8] features, which have also been found useful in many computer vision problems. These studies ( [5,6]) have proposed ways to preserve well-known handcrafted features through the compression process, without focusing on any particular problem. However, the recent trend in computer vision has been away from handcrafted features and towards learnt features, especially the features learnt by deep neural networks (DNNs) [9] for specific problems.…”