PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

Gong, Kehong; Li, Bingbing; Zhang, Jianfeng; Wang, Tao; Huang, Jing; Mi, Michael Bi; Feng, Jiashi; Wang, Xinchao

doi:10.1109/cvpr52688.2022.01074

Cited by 34 publications

(9 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we have focused purely on monocular video as this is the most widely available modality. We are also enthusiastic about integrating physics-based modeling in the inference process (42)(43)(44)(45) and ways to combine this with self-supervised learning (46). It is also important to make this system easier to use in a higher-throughput manner.…”

Section: Discussionmentioning

confidence: 99%

Portable in-clinic video-based gait analysis: validation study on prosthetic users

Anthony

Patel

Karakostas

et al. 2022

Preprint

View full text Add to dashboard Cite

Despite the common focus of gait in rehabilitation, there are few tools that allow quantitatively characterizing gait in the clinic. We recently described an algorithm, trained on a large dataset from our clinical gait analysis laboratory, which produces accurate cycle-by-cycle estimates of spatiotemporal gait parameters including step timing and walking velocity. Here, we demonstrate this system generalizes well to clinical care with a validation study on prosthetic users seen in therapy and outpatient clinic. Specifically, estimated walking velocity was similar to annotated 10-meter walking velocities, and cadence and foot contact times closely mirrored our wearable sensor measurements. Additionally, we found that a 2D keypoint detector pretrained on largely able-bodied individuals struggles to localize prosthetic joints, particularly for those individuals with more proximal or bilateral amputations, but it is possible to train a prosthetic-specific joint detector. Further work is required to validate the other outputs from our algorithm including sagittal plane joint angles and step length. Code and trained weights will be released upon publication.

show abstract

Section: Discussionmentioning

confidence: 99%

Portable in-clinic video-based gait analysis: validation study on prosthetic users

Anthony

Patel

Karakostas

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The traditional world-to-image projection typically involves transferring the world coordinates of the object to the camera space and then projecting them to the image plane. To simplify the process, previous methods [42,74,10,16,21,69] directly regress human poses in the camera coordinate system and feed them to a projection function. However, such a simplified process introduces an ambiguous problem applying the same projection function to different scenes.…”

Section: Camera and Motion Decouplingmentioning

confidence: 99%

“…Following previous baseline [42], we build these encoders upon temporal dilated convolutional networks. Moreover, inspired by previous skeleton-based 3D human pose estimation [31,42,74,10,66], we adopt two separate branches in the body-specific regressors to learn the global trajectory and root-relative body motions. Specifically, the trajectory branch learns the camera parameters α, ϕ, h and the global human transition t b , while the motion branch learns the shape parameter β b and pose parameter θ b of the body model.…”

Section: Sequential Full-body Motion Recoverymentioning

confidence: 99%

DeepFilter: A Deep Learning Based Variant Filter for VarDict

Zhang

Yin

Wei

et al. 2023

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

With the development of sequencing technologies, somatic mutation analysis has become an important component in cancer research and treatment. VarDict is a commonly used somatic variant caller for this task.Although the heuristic-based VarDict algorithm exhibits high sensitivity and versatility, it may detect higher amounts of false positive variants than callers, limiting its clinical practicality. To address this problem, we propose DeepFilter, a deep-learning based filter for VarDict, which can filter out the false positive variants detected by VarDict effectively.Our approach trains two models for insertion-deletion mutations (InDels) and single nucleotide variants (SNVs), respectively. Experiments show that DeepFilter can filter at least 98.5% of false positive variants and retain 93.5% of true positive variants for InDels and SNVs in the commonly used tumor-normal paired mode. Source code and pre-trained models are available at https://github.com/LeiHaoa/DeepFilter.

show abstract

“…Visual attention has been widely used in deep learning and achieves remarkable advances [59,69]. It has been exploited in computer vision tasks such as image recognition [9,10,33,48,61,71] and object detection among others [6,16,35,66,67]. CAM [77] provides the attention visualization of feature maps for model interpretable analysis.…”

Section: Related Workmentioning

confidence: 99%

Attention Diversification for Domain Generalization

Meng

Chen

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have demonstrated gratifying results at learning discriminative features. However, when applied to unseen domains, state-of-the-art models are usually prone to errors due to domain shift. After investigating this issue from the perspective of shortcut learning, we find the devils lie in the fact that models trained on different domains merely bias to different domain-specific features yet overlook diverse task-related features. Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features. Briefly, Intra-Model Attention Diversification Regularization is equipped on the high-level feature maps to achieve in-channel discrimination and cross-channel diversification via forcing different channels to pay their most salient attention to different spatial locations. Besides, Inter-Model Attention Diversification Regularization is proposed to further provide task-related attention diversification and domain-related attention suppression, which is a paradigm of "simulate, divide and assemble": simulate domain shift via exploiting multiple domain-specific models, divide attention maps into task-related and domain-related groups, and assemble them within each group respectively to execute regularization. Extensive experiments and analyses are conducted on various benchmarks to demonstrate that our method achieves state-of-the-art performance over other competing methods. Code is available at https: //github.com/hikvision-research/DomainGeneralization.

show abstract

PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

Cited by 34 publications

References 44 publications

Portable in-clinic video-based gait analysis: validation study on prosthetic users

Portable in-clinic video-based gait analysis: validation study on prosthetic users

DeepFilter: A Deep Learning Based Variant Filter for VarDict

Attention Diversification for Domain Generalization

Contact Info

Product

Resources

About