VI-Net—View-Invariant Quality of Human Movement Assessment

Sardari, Faegheh; Paiement, Adeline; Hannuna, Sion; Mirmehdi, Majid

doi:10.3390/s20185258

Cited by 22 publications

(21 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In United Kingdom an interest group called Digital and Informatics Physiotherapy Group (DIPG) (part of the Chartered Society of Physiotherapists, CSP) was formed to develop, evaluate and promote was formed to promote the use of novel technologies such as virtual reality (VR), telerehabilitation and AI in clinical physiotherapy. Having possibility of patients exercising at their homes with automatic real-time exercise feedback [161,173,209,294,477,538,595,674,675,704,815] could positively inuence exercise adherence leading to improved patient outcomes and lowered healthcare costs [445,884]. Similarly, these personalised and gamied approaches could be designed for general population as a preventive health measure [825], for example to increase the uptake of resistance training [63,867], as 80% of European adults do not meet the global resistance training guidelines of 2 or more days of resistance training per week [62].…”

Section: Precision Strength Trainingmentioning

confidence: 99%

Precision strength training: Data-driven artificial intelligence approach to strength and conditioning

Teikari¹,

Pietrusz²

2021

Preprint

View full text Add to dashboard Cite

In strength training, personalised strength training (autoregulation) approaches have been used to individualise exercise programs with monitoring an for dynamic adjustment based on their responses to training. While this transition from tradition-based training to evidence-based training framework has been an improvement in training practices, we argue that the future of strength training will also incorporate deep learning models powered by data. We refer to this data-driven framework as precision strength training inspired by the similar modeling frameworks used in precision medicine. In contrast to current personalised training in which the acquired athlete data is often subject to human expert decision-making, we are anticipating the rise of human-in-the-loop systems with an augmented coach who will be doing decisions collaboratively with the machine. Similar to other precision frameworks, such as precision health, we envision such a future to take decades to be realised and we focus here on practical short-term targets on a way to long-term realisation. In this chapter, we will review the measurement technology needed for continuous data acquisition from an individual during training/physical activity, how to acquire these datasets for the development of such systems and, how a proof-of-concept system could be developed for powerlifting training with applicability to general strength and conditioning (S&C) and physical rehabilitation purposes. Additionally, we will evaluate how the user experience (UX) of the system feedback and visualisation could be designed.

show abstract

Section: Precision Strength Trainingmentioning

confidence: 99%

Precision strength training: Data-driven artificial intelligence approach to strength and conditioning

Teikari¹,

Pietrusz²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In general, methods that rely on skeleton annotations must rely on fulsome 3D joint representations which are difficult to come by in in-the-wild scenarios. Recently a few works have developed view-invariant action recognition or analysis approaches from RGB-D images, such as [9,11,21,32]. Varol et al [9] deploy multi-view synthetic videos for training their network to perform action recognition given novel viewpoints, but still use 3D pose annotations to produce the synthetic data, while the newly generated videos would have to be also labelled by experts if they were to be used for specialist applications such as healthcare.…”

Section: Related Workmentioning

confidence: 99%

“…To tackle this problem, a simple solution would be to train a network on data from multiple views [9]. However, in practice, capturing a labelled dataset of different views is cumbersome and rare -but two example cases are the commonly used NTU [10] and the recent health-related QMAR [11] datasets, where the granularity of the labelling is still coarse at action class level and at overall performance score level respectively. Ideally, a wholly view-invariant approach would be trained on data from as few views as possible and be able to perform well on a single (unseen) view at inference time.…”

Section: Introductionmentioning

confidence: 99%

“…However, all these approaches rely on a significant amount of annotations (and sometimes camera parameters) during training, the provision of which is expensive and difficult in in-the-wild scenarios. Only very few works, such as [9,11,21] deal with training from RGB images only. Moreover, the authors are aware of no other unsupervised view-invariant RGB-only action classification or assessment study, and only of one such work based on RGB and depth [22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised View-Invariant Human Posture Representation

Sardari,

Ommer,

Mirmehdi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Most recent view-invariant action recognition and performance assessment approaches rely on a large amount of annotated 3D skeleton data to extract view-invariant features. However, acquiring 3D skeleton data can be cumbersome, if not impractical, in in-the-wild scenarios. To overcome this problem, we present a novel unsupervised approach that learns to extract view-invariant 3D human pose representation from a 2D image without using 3D joint data. Our model is trained by exploiting the intrinsic view-invariant properties of human pose between simultaneous frames from different viewpoints and their equivariant properties between augmented frames from the same viewpoint. We evaluate the learned view-invariant pose representations for two downstream tasks. We perform comparative experiments that show improvements on the state-of-the-art unsupervised cross-view action classification accuracy on NTU RGB+D by a significant margin, on both RGB and depth images. We also show the efficiency of transferring the learned representations from NTU RGB+D to obtain the first ever unsupervised cross-view and cross-subject rank correlation results on the multi-view human movement quality dataset, QMAR, and marginally improve on the-state-of-theart supervised results for this dataset. We also carry out ablation studies to examine the contributions of the different components of our proposed network.Figure 1: Left: the proposed network learns to disentangle canonical 3D human pose representations and viewdependent features through simultaneous frames from different views and augmented frames from the same view. Right: the unsupervised learned canonical pose representation can be used for downstream tasks.

show abstract

“…Spatiotemporal representations are densely packed with information regarding both the appearance and salient motion patterns occurring in the video clips, as illustrated in Figure 1. Due to this representational power, they are currently the best performing models on action-related tasks, such as action recognition [1][2][3][4], action quality assessment [5][6][7][8][9], skills assessment [10], and action detection [11]. This representation power comes at the cost of increased computational complexity [12][13][14][15], which makes 3D-CNNs unsuitable for deployment in resource-constrained scenarios.…”

Section: Introductionmentioning

confidence: 99%

HalluciNet-ing Spatiotemporal Representations Using a 2D-CNN

Parmar

Morris

2021

Signals

View full text Add to dashboard Cite

Spatiotemporal representations learned using 3D convolutional neural networks (CNN) are currently used in state-of-the-art approaches for action-related tasks. However, 3D-CNN are notorious for being memory and compute resource intensive as compared with more simple 2D-CNN architectures. We propose to hallucinate spatiotemporal representations from a 3D-CNN teacher with a 2D-CNN student. By requiring the 2D-CNN to predict the future and intuit upcoming activity, it is encouraged to gain a deeper understanding of actions and how they evolve. The hallucination task is treated as an auxiliary task, which can be used with any other action-related task in a multitask learning setting. Thorough experimental evaluation, it is shown that the hallucination task indeed helps improve performance on action recognition, action quality assessment, and dynamic scene recognition tasks. From a practical standpoint, being able to hallucinate spatiotemporal representations without an actual 3D-CNN can enable deployment in resource-constrained scenarios, such as with limited computing power and/or lower bandwidth. We also observed that our hallucination task has utility not only during the training phase, but also during the pre-training phase.

show abstract

VI-Net—View-Invariant Quality of Human Movement Assessment

Cited by 22 publications

References 46 publications

Precision strength training: Data-driven artificial intelligence approach to strength and conditioning

Precision strength training: Data-driven artificial intelligence approach to strength and conditioning

Unsupervised View-Invariant Human Posture Representation

HalluciNet-ing Spatiotemporal Representations Using a 2D-CNN

Contact Info

Product

Resources

About