A Convolutional Sequence to Sequence Model for Multimodal Dynamics Prediction in Ski Jumps

Zecha, Dan; Eggert, Christian; Einfalt, Moritz; Brehm, Stephan; Lienhart, Rainer

doi:10.1145/3265845.3265855

Cited by 14 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Parameter adaptation GMM Gesture-based interaction Gesture and Françoise et al, 2016 movement data Sarasua et al, 2016 Human-robot interaction Robot arm Calinon et al, 2007Calinon, 2016 One-shot HMM Gesture-based interaction Gesture and movement data Françoise and Bevilacqua, 2018 Incremental HMM Human-robot interaction 3D motion capture Kulić et al, 2008Kulić et al, , 2012 Stylistic HMM Movement synthesis 3D motion capture Tilmanne et al, 2012 Particle filtering Gesture-based interaction Gesture and movement data Caramiaux et al, 2015 Transfer 2017; Martinez et al, 2017;Kratzer et al, 2019;Wang and Feng, 2019), and Temporal or Spatio-temporal Convolutional Neural Networks (CNN) (Gehring et al, 2017;Li et al, 2018Li et al, , 2019Zecha et al, 2018).…”

Section: Type Of Adaptation Models Application Domains Input Data Papersmentioning

confidence: 99%

Machine Learning Approaches for Motor Learning: A Short Review

Caramiaux

Françoise

Liu

et al. 2020

Front. Comput. Sci.

View full text Add to dashboard Cite

Machine learning approaches have seen a considerable number of applications in human movement modeling but remain limited for motor learning. Motor learning requires that motor variability be taken into account and poses new challenges because the algorithms need to be able to differentiate between new movements and variation in known ones. In this short review, we outline existing machine learning models for motor learning and their adaptation capabilities. We identify and describe three types of adaptation: Parameter adaptation in probabilistic models, Transfer and meta-learning in deep neural networks, and Planning adaptation by reinforcement learning. To conclude, we discuss challenges for applying these models in the domain of motor learning support systems.

show abstract

Section: Type Of Adaptation Models Application Domains Input Data Papersmentioning

confidence: 99%

Machine Learning Approaches for Motor Learning: A Short Review

Caramiaux

Françoise

Liu

et al. 2020

Front. Comput. Sci.

View full text Add to dashboard Cite

show abstract

“…Their model is evaluated on hurdles and triple jump videos. [12] use a multi-step architecture to estimate the poses of ski jumpers. With a convolutional sequence to sequence model, they predict the jump forces of ski jumpers directly from the pose estimates.…”

Section: Related Workmentioning

confidence: 99%

“…In a previous system [12], the keypoint detection was split into separate steps. At first, the position of the ski jumper was located within the frame using MobileNet [19].…”

Section: Model Architecturementioning

confidence: 99%

Robust Estimation of Flight Parameters for SKI Jumpers

Ludwig

Einfalt

Lienhart

2020

2020 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

This paper presents a model that robustly estimates important flight parameters for ski jumpers during their flight phase based on several camera views from the side along the jumpers' typical flight trajectories. A convolutional neural network for pose estimation, but also trained to detect skis, serves as a base model. It identifies 98.0% of the relevant flight parameters correctly within an angle threshold of 5 degrees, improving by 11.6% over previous work. In postprocessing, a pose checker first removes all wrong poses by using comparisons of distances and relative positions of the detected keypoints. A second step executes two RANSAC variants. One robustly estimates the average pose and another one the average pose angles. This model lifts the detection performance to 99.3% of the relevant flight parameters within a threshold of 5 degrees.

show abstract

“…[19] use convolutional encoder-decoder networks on pose sequences -although only in 3D -to predict human motion forward in time. [35] consider 2D pose sequences for force estimation. Lastly, [25] use pose sequences in a multi-task CNN for human action recognition in videos.…”

Section: Related Workmentioning

confidence: 99%

“…The network therefore does not necessarily require a temporal receptive field covering the pose sequence of a complete video to infer the adjacent events during the mostly repetitive motion. We therefore limit the input of the network during training to a fixed sequence length s. No zero-padding is applied during convolutions, which otherwise has shown to have negative impact on sequence models [27,35]. We further do not pad the input sequences, since no events are annotated at the very beginning or end of a video.…”

Section: 22mentioning

confidence: 99%

Frame-Level Event Detection in Athletics Videos with Pose-Based Convolutional Sequence Networks

Einfalt

Dampeyrou

Zecha

et al. 2019

Proceedings Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports

Self Cite

View full text Add to dashboard Cite

In this paper we address the problem of automatic event detection in athlete motion for automated performance analysis in athletics. We specifically consider the detection of stride-, jump-and landing related events from monocular recordings in long and triple jump. Existing work on event detection in sports often uses manually designed features on body and pose configurations of the athlete to infer the occurrence of events. We present a two-step approach, where temporal 2D pose sequences extracted from the videos form the basis for learning an event detection model. We formulate the detection of discrete events as a sequence translation task and propose a convolutional sequence network that can accurately predict the timing of event occurrences. Our best performing architecture achieves a precision/recall of 92.3%/89.0% in detecting start and end of ground contact during the run-up and jump of an athlete at a temporal precision of ±1 frame at 200Hz. The results show that 2D pose sequences are a suitable motion representation for learning event detection in a sequence-to-sequence framework. CCS CONCEPTS• Computing methodologies → Visual content-based indexing and retrieval; Neural networks; Object detection. KEYWORDS event detection, video indexing, computer vision in sports, convolutional sequence modeling

show abstract

A Convolutional Sequence to Sequence Model for Multimodal Dynamics Prediction in Ski Jumps

Cited by 14 publications

References 12 publications

Machine Learning Approaches for Motor Learning: A Short Review

Machine Learning Approaches for Motor Learning: A Short Review

Robust Estimation of Flight Parameters for SKI Jumpers

Frame-Level Event Detection in Athletics Videos with Pose-Based Convolutional Sequence Networks

Contact Info

Product

Resources

About