Multimodal Deep Autoencoder for Human Pose Recovery

Abstract

Video-based human pose recovery is usually conducted by retrieving relevant poses using image features. In the retrieving process, the mapping between 2D images and 3D poses is assumed to be linear in most of the traditional methods. However, their relationships are inherently non-linear, which limits recovery performance of these methods. In this paper, we propose a novel pose recovery method using non-linear mapping with multi-layered deep neural network. It is based on feature extraction with multimodal fusion and back-propagation deep learning. In multimodal fusion, we construct hypergraph Laplacian with low-rank representation. In this way, we obtain a unified feature description by standard eigen-decomposition of the hypergraph Laplacian matrix. In back-propagation deep learning, we learn a non-linear mapping from 2D images to 3D poses with parameter fine-tuning. The experimental results on three data sets show that the recovery error has been reduced by 20%-25%, which demonstrates the effectiveness of the proposed method.

Computer assisted diagnosis of basal cell carcinoma using Z-transform features

“…Auto encoders are also used in some other computer vision tasks, such as human pose recovery and hand gesture recognition [31][32][33]. Authors in [31] proposed a three stage structure, consisting of two conventional auto encoders in the first and third stages and one multilayer neural network in the second stage for human pose recovery.…”

“…Authors in [31] proposed a three stage structure, consisting of two conventional auto encoders in the first and third stages and one multilayer neural network in the second stage for human pose recovery. To train the system, two kinds of inputs are used: 2D images (obtained from video frames) and 3D poses.…”

Establishing strong imputation performance of a denoising autoencoder in a wide range of missing data problems

“…An example of efficient usage of such autoencoder representations can be found for the task of human pose recovery. Hong et al [7] used an autoencoder architecture to reconstruct 3D poses from 2D silhouettes. Further algorithmic developments have introduced the denoising autoencoder [8] and the variational autoencoder [9], where the latter extends the autoencoder to a generative model.…”

A New Generalized Deep Learning Framework Combining Sparse Autoencoder and Taguchi Method for Novel Data Classification and Processing

“…However, proposed idea implements accuracies of 91% versus 86% and 93% versus 93% on the in vitro and in vivo datasets used in validating the US EPA method. Chaoqun Hong et al proposed a new pose retrieval technique which focuses on multimodal integration feature extraction and backpropagation deep neural network by using multilayered deep neural network with nonlinear mapping [9]. In [10] TzuHsi Song et al focused on bone marrow trepan biopsy images and proposed a hybrid deep autoencoder (HDA) network with Curvature Gaussian method for active and exact bone marrow hematopoietic stem cell detection via related highlevel feature correspondence.…”