Keypoint encoding and transmission for improved feature extraction from compressed images

Chao, Jianshu; Steinbach, Eckehard; Xie, Lexing

doi:10.1109/icme.2015.7177388

Cited by 2 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, the k i are the keypoints detected in I. Similar to our previous study on keypoint encoding for improved feature extraction from compressed images [19], we perform a simple experiment to demonstrate that the results of keypoint detection can be easily affected by video compression artifacts and that descriptors are more robust. To this end, we calculate the descriptors from the compressed video frames using the keypoints extracted from the original (uncompressed) frames.…”

Section: Sensitivity Of Keypoints and Descriptorsmentioning

confidence: 88%

“…In our previous study [19], we presented a keypoint encoding approach for still images. Applying this approach directly to individual frames in a video sequence would significantly increase the bitrate, as will be discussed in Section V. To address this issue, similar to the conventional inter-frame prediction scheme in video coding, we propose several keypoint prediction approaches that significantly reduce the number of keypoints to be encoded and thus the bitrate required for the side information.…”

Section: Core Ideasmentioning

confidence: 99%

“…• In Intra mode, the keypoints are extracted by conventional means from the current frame without reference to keypoints from previous frames and are encoded using a process similar to that presented in [19].…”

Section: B Keypoint Encodermentioning

confidence: 99%

“…Furthermore, a few approaches involve modifying standard image/video compression algorithms such that the features extracted from the compressed images/videos are as similar as possible to the features extracted from the uncompressed images/videos. These approaches (also belonging to the third class) are referred to as feature-preserving image and video compression [16]- [19]. To this end, the authors of [15], [16] optimize the JPEG quantization table, whereas in [17], [18], the rate allocation strategy is modified to preserve the most important features.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

Chao

Steinbach

2016

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

In many mobile visual analysis applications, compressed video is transmitted over a communication network and analyzed by a server. Typical processing steps performed at the server include keypoint detection, descriptor calculation, and feature matching. Video compression has been shown to have an adverse effect on feature-matching performance. The negative impact of compression can be reduced by using the keypoints extracted from the uncompressed video to calculate descriptors from the compressed video. Based on this observation, we propose to provide these keypoints to the server as side information and to extract only the descriptors from the compressed video. First, we introduce four different frame types for keypoint encoding to address different types of changes in video content. These frame types represent a new scene, the same scene, a slowly changing scene, or a rapidly moving scene and are determined by comparing features between successive video frames. Then, we propose Intra, Skip and Inter modes of encoding the keypoints for different frame types. For example, keypoints for new scenes are encoded using the Intra mode, and keypoints for unchanged scenes are skipped. As a result, the bitrate of the side information related to keypoint encoding is significantly reduced. Finally, we present pairwise matching and image retrieval experiments conducted to evaluate the performance of the proposed approach using the Stanford mobile augmented reality dataset and 720p format videos. The results show that the proposed approach offers significantly improved feature matching and image retrieval performance at a given bitrate.

show abstract

Section: Sensitivity Of Keypoints and Descriptorsmentioning

confidence: 88%

Section: Core Ideasmentioning

confidence: 99%

Section: B Keypoint Encodermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

Chao

Steinbach

2016

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, Chao and Steinbach proposed a JPEG-compatible compression scheme that preserved scale invariant feature transform (SIFT) features [2]. The second category is to extract features and directly encode the features rather than (or in addition to) the image content [3,4]. For example, compact descriptors for visual search (CDVS) had been standardized by MPEG [4]: the CDVS scheme extracts and encodes SIFT-like features for the purpose of content-based image retrieval, and it achieves satisfactory retrieval performance when using only 2 K or 4 K Bytes for a single image, much less bit-rate than using JPEG or JPEG2000.…”

Section: Related Workmentioning

confidence: 99%

Recognizable or Not: Towards Image Semantic Quality Assessment for Compression

Liu

Wang

2016

Sens Imaging

View full text Add to dashboard Cite

Traditionally, image compression was optimized for the pixel-wise fidelity or the perceptual quality of the compressed images given a bit-rate budget. But recently, compressed images are more and more utilized for automatic semantic analysis tasks such as recognition and retrieval. For these tasks, we argue that the optimization target of compression is no longer perceptual quality, but the utility of the compressed images in the given automatic semantic analysis task. Accordingly, we propose to evaluate the quality of the compressed images neither at pixel level nor at perceptual level, but at semantic level. In this paper, we make preliminary efforts towards image semantic quality assessment (ISQA), focusing on the task of optical character recognition (OCR) from compressed images. We propose a fullreference ISQA measure by comparing the features extracted from text regions of original and compressed images. We then propose to integrate the ISQA measure into an image compression scheme. Experimental results show that our proposed ISQA measure is much better than PSNR and SSIM in evaluating the semantic quality of compressed images; accordingly, adopting our ISQA measure to optimize compression for OCR leads to significant bit-rate saving compared to using PSNR or SSIM. Moreover, we perform subjective test about text recognition from compressed images, and observe that our ISQA measure has high consistency with subjective recognizability. Our work explores new dimensions in image quality assessment, and demonstrates promising direction to achieve higher compression ratio for specific semantic analysis tasks.

show abstract

Keypoint encoding and transmission for improved feature extraction from compressed images

Cited by 2 publications

References 17 publications

Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

Recognizable or Not: Towards Image Semantic Quality Assessment for Compression

Contact Info

Product

Resources

About