GPU-based MapReduce for large-scale near-duplicate video retrieval

Abstract

With the exponential growth of multimedia data, people are overwhelmed with massive amount of online videos, of which Near-Duplicate Videos (NDVs) occupy a large portion. In this paper, we present a novel framework for NDV retrieval, which explores the parallel power of two promising techniques: Graphics Processing Unit (GPU) and MapReduce. With the power of the proposed framework, various key algorithms in the field of computer vision, such as K-Means clustering, bag of features, inverted file index with hamming embedding and weak geometric consistency, are applied to NDV retrieval. Experimental results on the benchmark CC WEB VIDEO NDV dataset demonstrate that the proposed framework can significantly speed up processing huge amounts of video repositories.

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

“…Hence this approach is challenging to parallelize accurately and efficiently even for the stateof-the-art big-data frameworks. Wang et al [168] proposed a novel MapReduce framework called Multimedia and Intelligent Computing Cluster for near-duplicate video retrieval for large-scare multimedia data processing by joining the computing power of CPU's and GPU's to speed up the video data processing. They extract the keyframes using uniform sampling, store the keyframes to HDFS, perform local feature extraction using the Hessian-Affine detector [169] to detect interest points.…”

Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues

“…Hence this approach is challenging to parallelize accurately and efficiently even for the state-of-the-art big-data frameworks. Wang et al [168] proposed a novel MapReduce framework called Multimedia and Intelligent Computing Cluster for nearduplicate video retrieval for large-scare multimedia data processing by joining the computing power of CPU's and GPU's to speed up the video data processing. They extract the keyframes using uniform sampling, store the keyframes to HDFS, perform local feature extraction using the Hessian-Affine detector [169] to detect interest points.…”

Near-duplicate video detection based on an approximate similarity self-join strategy

“…In recent years, research involving video retrieval has been increased, mainly due to the popularity of sites for video sharing and viewing over the Web, e.g., YouTube. However, according to [1], several videos are either identical or almost identical when they are compared to each other. Usually, they differ in the format, encoding parameters, and the use of edition operations for the addition and/or removal of frames.…”

“…Some methods are also based on hash approaches to match similar videos and solve NDVR problem using global or local visual features as video representation [6]. As the amount of data grows, some works address the near-duplicate video detection problem by using parallel solutions such as MapReduce framework or GPU for large-scale NDVR problem [1]. When using the video-level strategy, the common approach to identify near-duplicate videos is based on the following: (i) videos are segmented into shots that are represented by key-frames; (ii) each key-frame is described by a signature (or descriptor), usually belonging to a high-dimensional space; (iii) then a global signature is computed over the key-frame descriptors to represent the whole video; and (iv) the similarity among videos can be computed by using either the set of signatures of key-frames or the global signature.…”