Unsupervised t-Distributed Video Hashing and Its Deep Hashing Extension

Abstract

Abstract-In this work, a novel unsupervised hashing algorithm, referred to as t-USMVH, and its extension to unsupervised deep hashing, referred to as t-UDH, are proposed to support large-scale video-to-video retrieval. To improve robustness of the unsupervised learning, t-USMVH combines multiple types of feature representations and effectively fuses them by examining a continuous relevance score based on a Gaussian estimation over pairwise distances, and also a discrete neighbor score based on the cardinality of reciprocal neighbors. To reduce sensitivity to scale changes for mapping objects that are far apart from each other, Student t-distribution is used to estimate the similarity between the relaxed hash code vectors for keyframes. This results in more accurate preservation of the desired unsupervised similarity structure in the hash code space. By adapting the corresponding optimization objective and constructing the hash mapping function via a deep neural network, we develop a robust unsupervised training strategy for a deep hashing network. The efficiency and effectiveness of the proposed methods are evaluated on two public video collections via comparisons against multiple classical and state-of-the-art methods.

Supervised Online Hashing via Hadamard Codebook Learning

“…The second kind (b) assigns a "codeword" to samples with the same class label, which heavily relies on the quality of codewords. The proposed SDOH aligns the distributions between the input data and the hashing space (c), which preserves similarity better in the produced Hamming space [22,9,23,10,34]. Different from the previous method [23], we present two effective and generalized distributions to solve both the imbalanced distribution and poor initialization problems.…”

“…Existing methods only preserve the information from the current data, and their update does not take the distributions of previous data into account. We argue that, these defects can be compensated by aligning the distributions between the input data and the hashing space when updating, which has been demonstrated informatively beyond online hashing as revealed in [22,9,23,10,34]. Inspired from it, we aim to impose an intuitive constraint on similarity preservation in the Hamming space to capture not only the pairwise similarity at the current stage, but also the semantic relationship among different stages.…”

Probabilistic Semantic Retrieval for Surveillance Videos With Activity Graphs

“…Video Hashing Methods: Hashing based methods focus on the computational needs of large-scale video retrieval by comparing a compact representation of query and archive videos learned from unsupervised or supervised data [1], [2], [31]. Yu et al [1] learn a hashing model based on extracting key frames and imposing pairwise constraints for semantically similar frames.…”

Finding Near-Duplicate Videos in Large-Scale Collections

“…Video-level approaches have been developed to deal with web-scale retrieval. In such approaches, videos are usually represented with a global signature such as an aggregated feature vector [55,34,16,39,6] or a hash code [45,46,12,11,23]. The video matching is based on the similarity computation between the video representations.…”