Human Motion Denoising Using Attention-Based Bidirectional Recurrent Neural Network

Kim, Seong Uk; Jang, Hye-Young; Kim, Jongmin

doi:10.1145/3355056.3364577

Cited by 6 publications

(7 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Building upon prior works [7], [14], [35], [53], machine learning-based methods [17], [18], [20], [23] for denoising motion data have been developed. Holden et al [17], [18] proposed a convolutional auto-encoder to learn the motion manifold and reconstruct corrupted motion data.…”

Section: Motion Denoisingmentioning

confidence: 99%

“…Holden et al [17], [18] proposed a convolutional auto-encoder to learn the motion manifold and reconstruct corrupted motion data. Kim et al [20] introduced a bidirectional recurrent neural network with an attention mechanism to improve denoising accuracy by emphasizing important input poses. Leng et al [23] proposed a method to estimate hand pose during tremors using a WaveNet and a graph neural network.…”

Section: Motion Denoisingmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

Zhou,

Shum,

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

In many human-computer interaction applications, fast and accurate hand tracking is necessary for an immersive experience. However, raw hand motion data can be flawed due to issues such as joint occlusions and high-frequency noise, hindering the interaction. Using only current motion for interaction can lead to lag, so predicting future movement is crucial for a faster response. Our solution is the Multi-task Spatial-Temporal Graph Auto-Encoder (Multi-STGAE), a model that accurately denoises and predicts hand motion by exploiting the inter-dependency of both tasks. The model ensures a stable and accurate prediction through denoising while maintaining motion dynamics to avoid over-smoothed motion and alleviate time delays through prediction. A gate mechanism is integrated to prevent negative transfer between tasks and further boost multi-task performance. Multi-STGAE also includes a spatial-temporal graph autoencoder block, which models hand structures and motion coherence through graph convolutional networks, reducing noise while preserving hand physiology. Additionally, we design a novel hand partition strategy and hand bone loss to improve natural hand motion generation. We validate the effectiveness of our proposed method by contributing two large-scale datasets with a data corruption algorithm based on two benchmark datasets. To evaluate the natural characteristics of the denoised and predicted hand motion, we propose two structural metrics. Experimental results show that our method outperforms the state-of-the-art, showcasing how the multitask framework enables mutual benefits between denoising and prediction.

show abstract

Section: Motion Denoisingmentioning

confidence: 99%

Section: Motion Denoisingmentioning

confidence: 99%

Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

Zhou,

Shum,

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Many deep learning methods [11,32,36] also focus on human motion data denoising and have achieved state-of-the-art results. In particular, Holden et al [9] try to learn motion manifolds with the convolutional auto-encoder.…”

Section: Denoising Neural Networkmentioning

confidence: 99%

“…Kim et al [11] have proposed a novel method of denoising human motion, using a bidirectional recurrent neural network with an attention mechanism. The attention mechanism ensures that a higher weight value is selectively given to the more important input at a specific frame, thus achieving better optimization results than others.…”

Section: Denoising Neural Networkmentioning

confidence: 99%

STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

Zhou

Cheng

Shum

et al. 2021

2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

“…The latest researches revealed that the data-driven methods also contributed to labelling. Kim [39] proposed a BRNN network with attention mechanism, which is also a deep-learning. Pavlo et al [40] proposed deeplearning-based finger animation model, which was a twostage pipeline based on deep neural network to reconstruct missing marker and repair joints deformation.…”

Section: B Labelling Maker Trajectorymentioning

confidence: 99%

Graph Matching for Marker Labeling and Missing Marker Reconstruction With Bone Constraint by LSTM in Optical Motion Capture

Xiao

2021

IEEE Access

View full text Add to dashboard Cite

Optical motion capture (MOCAP) is a commonly used technology to record the motion of non-rigid objects with high accuracy in 3D space. However, the MOCAP data has to be processed further before it can be used. The scattered reconstructed motion data must constitute a human configuration by labelling process according to the predefined template, and the missing markers have to be reconstructed to produce a stable motion trajectory. In this work, we propose a novel labelling method for motion sequences. First, a novel graph matching method is employed to determine the connection relationship of the scattered motion data for a single frame. Then, Kalman filtering is used for tracking in the motion sequence. As for the challenge coming from missing markers, we propose a new motion data preprocessing method considering the bone length constraint, which represents the information of variation in the relative position of adjacent markers. The processed motion data is input into a Long-Short Term Memory (LSTM) model to recover the missing markers and de-noise the motion data. The experiment conducted on our own dataset proves that our labelling method achieves a similar effect to Cortex, which is a commonly used commercial motion data analysis software. The experiment on CMU dataset demonstrates that our missing marker reconstruction method can achieve an art-of-state result. The labelling code will be pulished on https://github.com/Lijianfang6930/Graph-Matching-for-Marker-Labelling INDEX TERMS Data preprocessing; Graph matching; LSTM; MOCAP data.

show abstract

Human Motion Denoising Using Attention-Based Bidirectional Recurrent Neural Network

Cited by 6 publications

References 1 publication

Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

Graph Matching for Marker Labeling and Missing Marker Reconstruction With Bone Constraint by LSTM in Optical Motion Capture

Contact Info

Product

Resources

About