3D Pose Estimation Based on Reinforce Learning for 2D Image-Based 3D Model Retrieval

Nie, Weizhi; Jia, Wenwu; Li, Wenhui; Liu, An-An; Zhao, Sicheng

doi:10.1109/tmm.2020.2991532

Cited by 27 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D human pose estimation has received much attention by the research community and several new directions have emerged [8], [9], [10], [12], [23], [24], [25], [26]. A significant number of detectors exploit the much-developed 2D methods for 3D pose estimation.…”

Section: Related Workmentioning

confidence: 99%

Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework

Ghafoor¹,

Mahmood²

2022

Preprint

View full text Add to dashboard Cite

3D human pose estimation using monocular images is an important yet challenging task. Existing 3D pose detection methods exhibit excellent performance under normal conditions however their performance may degrade due to occlusion. Recently some occlusion aware methods have also been proposed, however, the occlusion handling capability of these networks has not yet been thoroughly investigated. In the current work, we propose an occlusion-guided 3D human pose estimation framework and quantify its occlusion handling capability by using different protocols. The proposed method estimates more accurate 3D human poses using 2D skeletons with missing joints as input. Missing joints are handled by introducing occlusion guidance that provides extra information about the absence or presence of a joint. Temporal information has also been exploited to better estimate the missing joints. A large number of experiments are performed for the quantification of occlusion handling capability of the proposed method on three publicly available datasets in various settings including random missing joints, fixed body parts missing, and complete frames missing, using mean per joint position error criterion. In addition to that, the quality of the predicted 3D poses is also evaluated using action classification performance as a criterion. 3D poses estimated by the proposed method achieved significantly improved action recognition performance in the presence of missing joints. Our experiments demonstrate the effectiveness of the proposed framework for handling the missing joints as well as quantification of the occlusion handling capability of the deep neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework

Ghafoor¹,

Mahmood²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, the research community pays a significant amount of attention to develop 3D pose estimation. Most of the methods [26], [27], [28], [29] on 3D pose estimation were originated from 2D pose estimation task. Some works [30], [31], [32], [33], [34], [35], [36] tried to develop multi-view based methods to get more accurate 3D pose estimation.…”

Section: Related Workmentioning

confidence: 99%

Wearable Motion Capture: Reconstructing and Predicting 3D Human Poses from Wearable Sensors

Moniruzzaman¹,

Yin²,

Hossain³

et al. 2021

Preprint

View full text Add to dashboard Cite

Identifying 3D human walking poses in unconstrained environments has many applications such as enabling prosthetists and clinicians to access the amputees’ walking functions outside clinics and helping amputees obtain an optimal walking condition with predictive control. Thus, we propose the wearable motion capture problem of reconstructing and predicting 3D human poses from the wearable IMU sensors and wearable cameras. To solve this challenging problem, we introduce a novel Attention-Oriented Recurrent Neural Network (AttRNet) that contains a sensor-wise attention-oriented recurrent encoder, a reconstruction module, and a dynamic temporal attention-oriented recurrent decoder, to reconstruct the current pose and predict the future poses. To evaluate our approach, we collected a new WearableMotionCapture dataset using wearable IMUs and wearable video cameras, along with the musculoskeletal joint angle ground truth. The proposed AttRNet shows high accuracy on theWearableMotionCapture dataset, and it also outperforms the current best methods on two public pose prediction datasets with IMU-only data: DIP-IMU and TotalCaputre. The source codes and the new dataset will be publicly available on https://github.com/MoniruzzamanMd/Wearable-Motion-Capture.<br>

show abstract

“…Lee et al [20] proposed a cross-view convolution to model a sequence of rendered views for each 3D object. To reduce the impact of view changes, Nie et al [33] proposed a 3D model pose estimation which can select 3D model pose when given a query, 2D image. Nie et al [34] proposed a multi-branch graph convolution network, which considers the correlation between 2D image and 3D model in the step of feature learning.…”

Section: Related Work 21 3d Object Retrievalmentioning

confidence: 99%

“…Facing the multi-modal big data, cross-domain 3D object retrieval is mandatory and dispensable to manage the big data. Most of previous methods aim to retrieve 3D objects with 3D object query [9][10][11], while (singletype) 2D image-based 3D object retrieval emerges recently [33][34][35]. For example, Liu et al [27] proposed 3D object retrieval method with visual domain adaptation.…”

Section: Introductionmentioning

confidence: 99%

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Song

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

2D image-based 3D object retrieval is a very important task in computer vision and big data management. Conventional image-based 3D object retrieval usually assumes that the images are from one single domain. However, for real applications, 2D images may be from multiple domains (e.g., real image, sketch, and quick draw). It raises significant challenges for this task since these 2D images have a great domain gap with each other as well as a great modality gap with 3D objects. To address these issues, we propose an unsupervised Domain-Specific Alignment Network (DSAN) for multidomain image-based 3D object retrieval. The proposed method aims to reduce domain discrepancy by domain-specific alignment network with multi-level moment matching, including first-order moment and second-order moment. Based on the observation that for any given sample, different domain classifiers should output the same label, we design a domain-specific classifier alignment module. To our knowledge, the proposed method is the first unsupervised work to align multiple-domain 2D images with 3D objects in an end-to-end manner. The multi-domain dataset MDI3D is utilized to advocate the research on this task, and the extensive experimental results demonstrate the superiority of the proposed method. CCS CONCEPTS • Computing methodologies → Computer vision; Visual contentbased indexing and retrieval; • Information systems → Multimedia and multimodal retrieval.

show abstract

3D Pose Estimation Based on Reinforce Learning for 2D Image-Based 3D Model Retrieval

Cited by 27 publications

References 41 publications

Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework

Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework

Wearable Motion Capture: Reconstructing and Predicting 3D Human Poses from Wearable Sensors

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Contact Info

Product

Resources

About