Deep Hashing for Motion Capture Data Retrieval

Abstract

In this work, we propose an efficient retrieval method for human motion capture (MoCap) data based on supervised deep hash code learning. Raw Mocap data is represented into three 2D images, which encode the trajectories, velocities and selfsimilarity of joints respectively. Such image-based representations are fed into a convolutional neural network (CNN) adapted from the pre-trained VGG16 network. Further, we add a hash layer to fine-tune the CNN and generate the hash codes. By minimizing the loss defined by classification error and constraints on hash codes, highly discriminative hash representations of the motion data can be generated. As experimentally demonstrated on the public HDM05 data set, our algorithm achieves high accuracy comparing with the state-ofthe-art MoCap data retrieval algorithms. Besides, it achieves high efficiency due to the fast matching of hash codes.

An Effective and Efficient Approach for 3D Recovery of Human Motion Capture Data

“…They also discuss a few future research directions for the management of large and diverse motion capture skeleton data. Lv et al [ 28 ] propose a hash-based convolution neural network where they extract deep features using the VGG16 network. They introduce the hash layer to create the hash code and, as a result, CNN is fine-tuned.…”

Deep Hash with Improved Dual Attention for Image Retrieval

“…It is common for deep hashing to be applied in data retrieval for its advantages of a solid learning ability and good portability [3]. Meanwhile, deep learning to hash methods [4][5][6][7][8][9][10][11] try to convert high-dimensional media data into compact binary code via a hash function, and the data structure information is stored in the Hamming space. Therefore, deep hashing methods garner attention in image retrieval.…”

Efficient Indexing of 3D Human Motions

“…Current research mainly focuses on recognizing classes of presegmented actions [5,10,12], detecting actions in a stream [15,23], or searching for query-relevant subsequences within a long motion [2,22]. These tasks often employ query-by-example retrieval as the underlying operation; e.g., in the subsequence search task, a long motion is usually partitioned into a large number of short motion segments that need to be effectively and efficiently matched against a user query.…”

“…Many existing retrieval techniques [2,18,19,24] focus solely on search quality and do not discuss the efficiency at all, which leads to expensive sequential scan over the whole dataset. The efficiencyoriented works either propose very compact features that allow fast sequential scanning [12,13], or utilize various indexing schemes to organize the motion data (e.g., the binary tree [25], kd tree [9], R* tree [4], inverted file index [14], or tries [8]). To optimize the efficiency-effectiveness trade-off, a two-phase retrieval model is often used, where the candidate objects identified within an efficient search phase are submitted to a re-ranking phase that refines the result using more expensive techniques (e.g., traversal of a graph structure [9] or ranking by the Dynamic Time Warping [14,20]).…”