AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training

Fieraru, Mihai; Zanfir, Mihai; Pirlea, Silviu Cristian; Olaru, Vlad; Sminchisescu, Cristian

doi:10.1109/cvpr46437.2021.00979

Cited by 48 publications

(17 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, MotionMA [49] and ReactiveVideo [11] align the experts' poses captured by Kinect onto the novices' poses in videos to visualize the difference in postures. AIFit [13] mines and highlights the most significantly different features from the comparisons of reconstructed 3D poses from videos. Even though AIFit is fully automatic, the dominant differences might not reflect informative feedback to the sport.…”

Section: Related Workmentioning

confidence: 99%

“…In visualization, the shapes of graphical annotation markers overlaid on videos are also subject to changing viewpoints, and are thus ambiguous in providing accurate corrective feedback to be perceived by amateur runners. To promote spatial awareness, prior studies have attempted to analyze reconstructed 3D poses [13], fuse videos in multiviews [46], and use situated AR [28] and immersive visualization [10,24]. Thanks to the emerging methods in monocular human reconstruction in computer vision [9,16], reconstructing 3D poses has become an effective and accessible solution for videos.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

VCoach: A Customizable Visualization and Analysis System for Video-based Running Coaching

Liu¹,

Saquib²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

VCoach: A Customizable Visualization and Analysis System for Video-based Running Coaching

Liu¹,

Saquib²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…Moreover, most of existing works operate on 2D pose inputs [1,2,3,6,5]. Similar to [4], we also design our framework to work with 3D poses enabling us to be robust to ambiguities found in 2D poses. While a few works took some steps toward giving feedback [2,3,4], this was achieved in a hard-coded fashion, by thresholding angles between some of the body joints.…”

Section: Physical Exercise Analysismentioning

confidence: 99%

“…There is therefore a growing need for computer-aided exercise feedback strategies. A few recent works have addressed this problem [1,2,3,4,5,6]. However, they focus only on identifying whether an exercise is performed correctly or not [1,6], or they rely on hard-coded rules based on joint angles that cannot easily be extended to new exercises [2,3,4].…”

Section: Introductionmentioning

confidence: 99%

“…A few recent works have addressed this problem [1,2,3,4,5,6]. However, they focus only on identifying whether an exercise is performed correctly or not [1,6], or they rely on hard-coded rules based on joint angles that cannot easily be extended to new exercises [2,3,4]. In this work, we therefore leverage recent advances in the fields of pose estimation [7,8,9], action recognition [10,11] and motion prediction [12,13,14] to design a framework that provides automated and personalized feedback to supervise physical exercises.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

3D Pose Based Feedback for Physical Exercises

Zhao¹,

Kiciroglu²,

Hugues³

et al. 2022

Preprint

View full text Add to dashboard Cite

Unsupervised self-rehabilitation exercises and physical training can cause serious injuries if performed incorrectly. We introduce a learning-based framework that identifies the mistakes made by a user and proposes corrective measures for easier and safer individual training. Our framework does not rely on hard-coded, heuristic rules. Instead, it learns them from data, which facilitates its adaptation to specific user needs. To this end, we use a Graph Convolutional Network (GCN) architecture acting on the user's pose sequence to model the relationship between the the body joints trajectories. To evaluate our approach, we introduce a dataset with 3 different physical exercises. Our approach yields 90.9% mistake identification accuracy and successfully corrects 94.2% of the mistakes.

show abstract

PoseScript: 3D Human Poses from Natural Language

Ginger

Weinzaepfel²,

Lucas³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Automatically producing instructions to modify one's posture could open the door to endless applications, such as personalized coaching and in-home physical therapy. Tackling the reverse problem (i.e., refining a 3D pose based on some natural language feedback) could help for assisted 3D character animation or robot teaching, for instance. Although a few recent works explore the connections between natural language and 3D human pose, none focus on describing 3D body pose differences. In this paper, we tackle the problem of correcting 3D human poses with natural language. To this end, we introduce the Pose-Fix dataset, which consists of several thousand paired 3D poses and their corresponding text feedback, that describe how the source pose needs to be modified to obtain the target pose. We demonstrate the potential of this dataset on two tasks: (1) text-based pose editing, that aims at generating corrected 3D body poses given a query pose and a text modifier; and (2) correctional text generation, where instructions are generated based on the differences between two body poses. The dataset and the code are available at https://europe.naverlabs.com/research/ computer-vision/posefix/.

show abstract

AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training

Cited by 48 publications

References 33 publications

VCoach: A Customizable Visualization and Analysis System for Video-based Running Coaching

VCoach: A Customizable Visualization and Analysis System for Video-based Running Coaching

3D Pose Based Feedback for Physical Exercises

PoseScript: 3D Human Poses from Natural Language

Contact Info

Product

Resources

About