Temporal 3D Human Pose Estimation for Action Recognition from Arbitrary Viewpoints

Musallam, Mohamed Adel; Baptista, Renato; Ismaeil, Kassem Al; Aouada, Djamila

doi:10.1109/csci49370.2019.00052

Cited by 3 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, these two tasks are closely related. Every 3D pose can be projected to a 2D pose, and a 3D pose can also be inferred using 2D pose estimation [ 48 ]. Most current Human Pose Estimation algorithms are focused on predicting the coordinates of human keypoint, i.e., keypoint localization, which portrays the human pose by determining the spatial location relationship between keypoints through a priori knowledge.…”

Section: Related Workmentioning

confidence: 99%

Identity-Preserved Human Posture Detection in Infrared Thermal Images: A Benchmark

Guo

Chen

Deng

et al. 2022

Sensors

View full text Add to dashboard Cite

Human pose estimation has a variety of real-life applications, including human action recognition, AI-powered personal trainers, robotics, motion capture and augmented reality, gaming, and video surveillance. However, most current human pose estimation systems are based on RGB images, which do not seriously take into account personal privacy. Although identity-preserved algorithms are very desirable when human pose estimation is applied to scenarios where personal privacy does matter, developing human pose estimation algorithms based on identity-preserved modalities, such as thermal images concerned here, is very challenging due to the limited amount of training data currently available and the fact that infrared thermal images, unlike RGB images, lack rich texture cues which makes annotating training data itself impractical. In this paper, we formulate a new task with privacy protection that lies between human detection and human pose estimation by introducing a benchmark for IPHPDT (i.e., Identity-Preserved Human Posture Detection in Thermal images). This task has a threefold novel purpose: the first is to establish an identity-preserved task with thermal images; the second is to achieve more information other than the location of persons as provided by human detection for more advanced computer vision applications; the third is to avoid difficulties in collecting well-annotated data for human pose estimation in thermal images. The presented IPHPDT dataset contains four types of human postures, consisting of 75,000 images well-annotated with axis-aligned bounding boxes and postures of the persons. Based on this well-annotated IPHPDT dataset and three state-of-the-art algorithms, i.e., YOLOF (short for You Only Look One-level Feature), YOLOX (short for Exceeding YOLO Series in 2021) and TOOD (short for Task-aligned One-stage Object Detection), we establish three baseline detectors, called IPH-YOLOF, IPH-YOLOX, and IPH-TOOD. In the experiments, three baseline detectors are used to recognize four infrared human postures, and the mean average precision can reach 70.4%. The results show that the three baseline detectors can effectively perform accurate posture detection on the IPHPDT dataset. By releasing IPHPDT, we expect to encourage more future studies into human posture detection in infrared thermal images and draw more attention to this challenging task.

show abstract

Section: Related Workmentioning

confidence: 99%

Identity-Preserved Human Posture Detection in Infrared Thermal Images: A Benchmark

Guo

Chen

Deng

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Estimating the 3D location from individual frames leads to a temporally incoherent result, where the independent error from each frame leads to unstable 3D position estimation over the video sequence. Thus, in our work, we follow the same approach proposed in [22,17] for human pose estimation where a fully convolutional architecture is used to perform temporal convolution over 2D skeleton joint positions in order to estimate the 3D skeleton in a video. Therefore, the function g(•) is approximated by a Sequence to Sequence (Seq2Seq) Temporal Convolutional Network (TCN) model as can be seen in Figure 4 using 1D temporal convolution.…”

Section: D Trajectory Estimationmentioning

confidence: 99%

Leveraging Temporal Information for 3D Trajectory Estimation of Space Objects

Musallam

Castillo

Ismaeil

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

Self Cite

View full text Add to dashboard Cite

This work presents a new temporally consistent space object 3D trajectory estimation from a video taken by a single RGB camera. Understanding space objects' trajectories is an important component of Space Situational Awareness, especially for applications such as Active Debris Removal, On-orbit Servicing, and Orbital Maneuvers. Using only the information from a single image perspective gives temporally inconsistent 3D position estimation. Our approach operates in two subsequent stages. The first stage estimates the 2D location of the space object using a convolution neural network. In the next stage, the 2D locations are lifted to 3D space, using temporal convolution neural network that enforces the temporal coherence over the estimated 3D locations. Our results show that leveraging temporal information yields smooth and accurate 3D trajectory estimations for space objects. A dedicated large realistic synthetic dataset, named SPARK-T, containing 3 spacecrafts, under various sensing conditions, is also proposed and will be publicly shared with the research community.

show abstract