Adaptive Action Assessment

Pan, Jiahui; Gao, Jibin; Zheng, Wei‐Shi

doi:10.1109/tpami.2021.3126534

Cited by 15 publications

(1 citation statement)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…loss AQA = loss sv + loss f s + loss at (10) Once the training stage is completed, the optimal parameters of the feature extraction module and the score prediction module can be determined. During the testing stage, only one branch of the AQA network needs to be used for inference.…”

mentioning

confidence: 99%

A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises

Yao,

Lei,

Zhang

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Human activity analysis in the legal monitoring environment plays an important role in the physical rehabilitation field, as it helps patients with physical injuries improve their postoperative conditions and reduce their medical costs. Recently, several deep learning-based action quality assessment (AQA) frameworks have been proposed to evaluate physical rehabilitation exercises. However, most of them treat this problem as a simple regression task, which requires both the action instance and its score label as input. This approach is limited by the fact that the annotations in this field usually consist of healthy or unhealthy labels rather than quality scores provided by professional physicians. Additionally, most of these methods cannot provide informative feedback on a patient's motion defects, which weakens their practical application. To address these problems, we propose a multi-task contrastive learning framework to learn subtle and critical differences from skeleton sequences to deal with the performance metric and AQA problems of physical rehabilitation exercises. Specifically, we propose a performance metric network that takes triplets of training samples as input for score generation. For the AQA task, the same contrast learning strategy is used, but pairwise training samples are fed into the action quality assessment network for score prediction. Notably, we propose quantifying the deviation of the joint attention matrix between different skeleton sequences and introducing it into the loss function of our learning network. It is proven that considering both score prediction loss and joint attention deviation loss improves physical exercises AQA performance. Furthermore, it helps to obtain informative feedback for patients to improve their motion defects by visualizing the joint attention matrix's difference. The proposed method is verified on the UI-PRMD and KIMORE datasets. Experimental results show that the proposed method achieves state-of-the-art performance.

show abstract

mentioning

confidence: 99%

A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises

Yao,

Lei,

Zhang

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

Pairwise Contrastive Learning Network for Action Quality Assessment

Li¹,

Lei²,

Fan³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Label-reconstruction-based pseudo-subscore learning for action quality assessment in sporting events

et al. 2022

View full text Add to dashboard Cite

Most existing action quality assessment (AQA) methods provide only an overall quality score for the input video and lack an evaluation of each substage of the movement process; thus, these methods cannot provide detailed feedback for users. Moreover, the existing datasets do not provide labels for substage quality assessment. To address these problems, in this work, a new label-reconstruction-based pseudo-subscore learning (PSL) method is proposed for AQA in sporting events. In the proposed method, the overall score of an action is not only regarded as a quality label but also used as a feature of the training set. A label-reconstruction-based learning algorithm is built to generate pseudo-subscore labels for the training set. Moreover, based on the pseudo-subscore labels and overall score labels, a multi-substage AQA model is fine-tuned from the PSL model to predict the action quality score of each substage and the overall score for an athlete. Several ablation experiments are performed to verify the effectiveness of each module. The experimental results show that our approach achieves state-of-the-art performance.

show abstract

Adaptive Action Assessment

Cited by 15 publications

References 56 publications

A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises

A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises

Pairwise Contrastive Learning Network for Action Quality Assessment

Label-reconstruction-based pseudo-subscore learning for action quality assessment in sporting events

Contact Info

Product

Resources

About