Fast and accurate low bit rate retrieval-by-capture applications

Abstract

Retrieval-by-capture applications allow a user to scan a printed picture in a newspaper or magazine with a mobile camera to then automatically trigger the display of predetermined related content. To ensure a high user Quality of Experience (QoE), this paper proposes a fast and accurate low bit rate solution for retrieval-by-capture applications based on extracting Scale Invariant Feature Transform (SIFT) features from images reconstructed from the low spatial frequency components. The system applies a two-dimensional block-based Discrete Cosine Transform (DCT), as typically used in image coders, and only encodes and transmits the resulting DC components. Received images are decoded using these components and SIFT features are derived and then used to find a matching image within a database of candidate images. Results show that the proposed system achieves more than 97% matching accuracy when evaluated for a wide range of typical image distortions including scaling, rotation, additive noise, image blurring and illumination. Transmission data rates are comparable to existing compressed domain image features whilst significantly reducing system latency.