High-speed multi-person pose estimation with deep feature transfer

Huang, Ying; Shum, Hubert P. H.; Ho, Edmond S. L.; Aslam, Nauman

doi:10.1016/j.cviu.2020.103010

Cited by 7 publications

(5 citation statements)

References 44 publications

(85 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By incorporating part-based assessment, the potential to provide further analytic cues for CP prediction and the identification of CP sub-types is enhanced [43]. As such, alternative methods have been proposed to better model human shape and motion, such as pose estimation [9], [22].…”

Section: B Automated Optical Flow-based Methodsmentioning

confidence: 99%

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants

McCay

Shum

et al. 2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using posebased features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework's classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

show abstract

Section: B Automated Optical Flow-based Methodsmentioning

confidence: 99%

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants

McCay

Shum

et al. 2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…Some works also use neural networks to learn the parameters of a 3DMM model for face recovery, which combines the advantage of neural networks in parameter fitting and the 3DMM model in recovering stable geometric and textural representations [ 6 , 7 ]. The advancement of deep learning solutions, however, comes with the requirement of more training data [ 35 , 36 , 37 , 38 , 39 ]. Due to the high cost of acquiring 3D face scanning data, some methods that rely on weekly-supervised learning have also been proposed to reconstruct faces through synthesis by analysis [ 6 , 9 , 17 , 34 ].…”

Section: Related Workmentioning

confidence: 99%

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Huang

Lin

2023

Sensors

Self Cite

View full text Add to dashboard Cite

We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

show abstract

“…The consistency of several elements throughout time can be used to assess a joint's reliability, and learning the weights of dependability terms improves the performance of the classifier [14]. To capture the concurrent correlations between body joint and limb data, Huang et al suggested a feature transfer paradigm [15]. Concurrent correlations of these features can help build a structural link that might improve the network's inferences while reducing the requirement for refinement modules.…”

Section: Related Workmentioning

confidence: 99%

Computer-Aided Depth Video Stream Masking Framework for Human Body Segmentation in Depth Sensor Images

Ryselis

Blažauskas

Damaševičius

et al. 2022

Sensors

View full text Add to dashboard Cite

The identification of human activities from videos is important for many applications. For such a task, three-dimensional (3D) depth images or image sequences (videos) can be used, which represent the positioning information of the objects in a 3D scene obtained from depth sensors. This paper presents a framework to create foreground–background masks from depth images for human body segmentation. The framework can be used to speed up the manual depth image annotation process with no semantics known beforehand and can apply segmentation using a performant algorithm while the user only adjusts the parameters, or corrects the automatic segmentation results, or gives it hints by drawing a boundary of the desired object. The approach has been tested using two different datasets with a human in a real-world closed environment. The solution has provided promising results in terms of reducing the manual segmentation time from the perspective of the processing time as well as the human input time.

show abstract

High-speed multi-person pose estimation with deep feature transfer

Cited by 7 publications

References 44 publications

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Computer-Aided Depth Video Stream Masking Framework for Human Body Segmentation in Depth Sensor Images

Contact Info

Product

Resources

About