Human Pose and Shape Estimation From Single Polarization Images

Zou, Shihao; Zuo, Xinxin; Wang, Sen; Qian, Yiming; Guo, Chuan; Ji, Wei; Li, Jingjing; Gong, Maoguo; Cheng, Li

doi:10.1109/tmm.2022.3162469

Cited by 14 publications

(10 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jointly using the Snell and Fresnel theories, they revert the ray bending caused by the change in medium and handle the problem as if the measurements were done outside the water. Zou et al 53 pushed forward the accuracy in the human shape and pose estimation by building a two steps network with polarization cues. Assuming the human cloth to be diffuse dominant, they retrieve the human features by using the raw polarization intensity images, and the ambiguous normal maps obtained from Fresnel theory.…”

Section: Pose Estimationmentioning

confidence: 99%

Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

Rodriguez,

Lew-Yan-Voon,

Martins

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Polarization information of the light can provide rich cues for computer vision and scene understanding tasks, such as the type of material, pose, and shape of the objects. With the advent of new and cheap polarimetric sensors, this imaging modality is becoming accessible to a wider public for solving problems, such as pose estimation, 3D reconstruction, underwater navigation, and depth estimation. However, we observe several limitations regarding the usage of this sensorial modality, as well as a lack of standards and publicly available tools to analyze polarization images. Furthermore, although polarization camera manufacturers usually provide acquisition tools to interface with their cameras, they rarely include processing algorithms that make use of the polarization information. In this work, we review recent advances in applications that involve polarization imaging, including a comprehensive survey of recent advances on polarization for vision and robotics perception tasks. We also introduce a complete software toolkit that provides common standards to communicate with and process information from most of the existing micro-grid polarization cameras on the market. The toolkit also implements several image processing algorithms for this modality, and it is publicly available on GitHub.

show abstract

Section: Pose Estimationmentioning

confidence: 99%

Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

Rodriguez,

Lew-Yan-Voon,

Martins

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Although the dataset provides more than 4.5 hours event stream and 21 different types of action, it may still lack sufficient degree of pose variety, partly due to its in-house capturing setup. In this work, we significantly augment the MMPHSPD dataset by synthesizing events data from several human motion capture benchmark datasets [29], [51]- [53] -it gives rise to a large-scale dataset, SynEventHPD, containing a rich variety of poses for the task of event-based human pose tracking. This dataset will be instrumental in further facilitating the research on event-based vision and SNNs for human pose tracking.…”

Section: Related Workmentioning

confidence: 99%

“…To address this issue, we propose to synthesize event data from multiple motion capture datasets, including i.e. Human3.6M [51], AMASS [52], PHSPD [53] and MMHPSD-Gray [29], to construct a large-scale synthetic dataset. Our synthetic dataset, called SynEventHPD, is a meta dataset consisting of 4 subdatasets: EventH36M, EventAMASS, EventPHSPD and Syn-MMHPSD.…”

Section: Our Large-scale Synthetic Syneventhpd Datasetmentioning

confidence: 99%

“…We convert the high frame rate videos to gray-scale images and generate events by checking the brightness change at each pixel over time. This process is straightforward for Human3.6M [51], PHSPD [53] and MMHPSD [29], as they contain RGB or gray-scale videos. However, AMASS [52] dataset only contains motion capture data without any RGB videos.…”

Section: Our Large-scale Synthetic Syneventhpd Datasetmentioning

confidence: 99%

“…It consists of synthesized events data from multiple motion capture datasets (i.e. Human3.6M [51], AMASS [52], PHSPD [53] and MMHPSD-Gray [29]).…”

mentioning

confidence: 99%

See 2 more Smart Citations

EventHPE: Event-based 3D Human Pose and Shape Estimation

Zou

Guo

Zuo

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Event camera, as an emerging biologically-inspired vision sensor for capturing motion dynamics, presents new potential for 3D human pose tracking, or video-based 3D human pose estimation. However, existing works in pose tracking either require the presence of additional gray-scale images to establish a solid starting pose, or ignore the temporal dependencies all together by collapsing segments of event streams to form static event frames. Meanwhile, although the effectiveness of Artificial Neural Networks (ANNs, a.k.a. dense deep learning) has been showcased in many event-based tasks, the use of ANNs tends to neglect the fact that compared to the dense frame-based image sequences, the occurrence of events from an event camera is spatiotemporally much sparser. Motivated by the above mentioned issues, we present in this paper a dedicated end-to-end sparse deep learning approach for event-based pose tracking: 1) to our knowledge this is the first time that 3D human pose tracking is obtained from events only, thus eliminating the need of accessing to any frame-based images as part of input; 2) our approach is based entirely upon the framework of Spiking Neural Networks (SNNs), which consists of Spike-Element-Wise (SEW) ResNet and a novel Spiking Spatiotemporal Transformer; 3) a large-scale synthetic dataset is constructed that features a broad and diverse set of annotated 3D human motions, as well as longer hours of event stream data, named SynEventHPD. Empirical experiments demonstrate that, with superior performance over the state-of-the-art (SOTA) ANNs counterparts, our approach also achieves a significant computation reduction of 80% in FLOPS. Furthermore, our proposed method also outperforms SOTA SNNs in the regression task of human pose tracking. Our implementation is available at https://github.com/JimmyZou/HumanPoseTracking SNN and dataset will be released upon paper acceptance.

show abstract