Predicting the What and How - a Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling

Bütepage, Judith; Kragić, Danica

doi:10.1109/cvprw.2019.00352

Cited by 11 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human motion prediction is a complex task due to the inherent uncertainty in forecasting into the future. In recent years, many deep learning methods based on recurrent neural networks (RNNs) [15,16,17,18,19,20,14], variational auto encoders (VAEs) [21,22,23,24], transformers [12], and graph convolutional networks (GCNs) [25,26,13,27] have been proposed. We focus our discussion on GCN based ones, as we also exploit the graphlike connections of human joints with GCNs in our approach.…”

Section: Human Motion Predictionmentioning

confidence: 99%

3D Pose Based Feedback for Physical Exercises

Zhao¹,

Kiciroglu²,

Hugues³

et al. 2022

Preprint

View full text Add to dashboard Cite

Unsupervised self-rehabilitation exercises and physical training can cause serious injuries if performed incorrectly. We introduce a learning-based framework that identifies the mistakes made by a user and proposes corrective measures for easier and safer individual training. Our framework does not rely on hard-coded, heuristic rules. Instead, it learns them from data, which facilitates its adaptation to specific user needs. To this end, we use a Graph Convolutional Network (GCN) architecture acting on the user's pose sequence to model the relationship between the the body joints trajectories. To evaluate our approach, we introduce a dataset with 3 different physical exercises. Our approach yields 90.9% mistake identification accuracy and successfully corrects 94.2% of the mistakes.

show abstract

Section: Human Motion Predictionmentioning

confidence: 99%

3D Pose Based Feedback for Physical Exercises

Zhao¹,

Kiciroglu²,

Hugues³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead of top-down approaches like explicit cost functions or target-specific training data, the authors used a bottom-up, data-driven model that was trained in an unsupervised way. Knowing regularities in the way humans move allows the controller to make predictions about the human’s actions, which greatly limits the space of possible robot movement trajectories and thereby lowers response times ( Bütepage et al, 2019 ). It has to be pointed out that this approach is different from gesture recognition in that it does not attempt to derive abstract descriptions of the movements like pointing or stirring, which is then the basis for decision making and action planning.…”

Section: Relevance For Human–robot Interactionmentioning

confidence: 99%

Socializing Sensorimotor Contingencies

Lübbert

Göschl

Krause

et al. 2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

The aim of this review is to highlight the idea of grounding social cognition in sensorimotor interactions shared across agents. We discuss an action-oriented account that emerges from a broader interpretation of the concept of sensorimotor contingencies. We suggest that dynamic informational and sensorimotor coupling across agents can mediate the deployment of action-effect contingencies in social contexts. We propose this concept of socializing sensorimotor contingencies (socSMCs) as a shared framework of analysis for processes within and across brains and bodies, and their physical and social environments. In doing so, we integrate insights from different fields, including neuroscience, psychology, and research on human–robot interaction. We review studies on dynamic embodied interaction and highlight empirical findings that suggest an important role of sensorimotor and informational entrainment in social contexts. Furthermore, we discuss links to closely related concepts, such as enactivism, models of coordination dynamics and others, and clarify differences to approaches that focus on mentalizing and high-level cognitive representations. Moreover, we consider conceptual implications of rethinking cognition as social sensorimotor coupling. The insight that social cognitive phenomena like joint attention, mutual trust or empathy rely heavily on the informational and sensorimotor coupling between agents may provide novel remedies for people with disturbed social cognition and for situations of disturbed social interaction. Furthermore, our proposal has potential applications in the field of human–robot interaction where socSMCs principles might lead to more natural and intuitive interfaces for human users.

show abstract

“…Based on CVAE Kragic et. al [27] have proposed semi-supervised recurrent neural network(SVRNN) to detect or classify and predict human pose. Aliakbarian et.…”

Section: B Feed-forward Approaches Forecasts Human Motionmentioning

confidence: 99%

Development of Human Motion Prediction Strategy using Inception Residual Block

Gupta,

Yadav,

Nandi

2021

Preprint

View full text Add to dashboard Cite

Human Motion Prediction is a crucial task in computer vision and robotics. It has versatile application potentials such as in the area of human-robot interactions, human action tracking for airport security systems, autonomous car navigation, computer gaming to name a few. However, predicting human motion based on past actions is an extremely challenging task due to the difficulties in detecting spatial and temporal features correctly. To detect temporal features in human poses, we propose an Inception Residual Block(IRB), due to its inherent capability of processing multiple kernels to capture salient features. Here, we propose to use multiple 1-D Convolution Neural Network (CNN) with different kernel sizes and input sequence lengths and concatenate them to get proper embedding. As kernels strides over different receptive fields, they detect smaller and bigger salient features at multiple temporal scales. Our main contribution is to propose a residual connection between input and the output of the inception block to have a continuity between the previously observed pose and the next predicted pose. With this proposed architecture, it learns prior knowledge much better about human poses and we achieve much higher prediction accuracy as detailed in the paper. Subsequently, we further propose to feed the output of the inception residual block as an input to the Graph Convolution Neural Network (GCN) due to its better spatial feature learning capability. We perform a parametric analysis for better designing of our model and subsequently, we evaluate our approach on the Human 3.6M dataset and compare our short-term as well as long-term predictions with the state of the art papers, where our model outperforms most of the pose results, the detailed reasons of which have been elaborated in the paper.

show abstract

Predicting the What and How - a Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling

Cited by 11 publications

References 32 publications

3D Pose Based Feedback for Physical Exercises

3D Pose Based Feedback for Physical Exercises

Socializing Sensorimotor Contingencies

Development of Human Motion Prediction Strategy using Inception Residual Block

Contact Info

Product

Resources

About