Real-Time Human Body Pose Estimation for In-Car Depth Images

Torres, Helena R.; Oliveira, Bruno; Fonseca, Jaime C.; Queirós, Sandro; Borges, João; Rodrigues, Nélson; Coelho, Victor; Pallauf, Johannes; Brito, José Henrique Silveira de; Mendes, J.

doi:10.1007/978-3-030-17771-3_14

Cited by 6 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding to the works on the in-car environment, the work proposed by Torres et al [ 15 ] can detect the human body pose inside the vehicle by using a time-of-flight sensor which that could provide more light immunity compared to RGB sensors. In [ 16 ], Dixe et al use a similar approach to detect multi-person human body detection by using depth images generated synthectically with the knowledge of the work developed by Borges et al in [ 17 , 18 ].…”

Section: Related Workmentioning

confidence: 99%

Fusion Object Detection and Action Recognition to Predict Violent Action

Rodrigues

Costa

Melo

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In the context of Shared Autonomous Vehicles, the need to monitor the environment inside the car will be crucial. This article focuses on the application of deep learning algorithms to present a fusion monitoring solution which was three different algorithms: a violent action detection system, which recognizes violent behaviors between passengers, a violent object detection system, and a lost items detection system. Public datasets were used for object detection algorithms (COCO and TAO) to train state-of-the-art algorithms such as YOLOv5. For violent action detection, the MoLa InCar dataset was used to train on state-of-the-art algorithms such as I3D, R(2+1)D, SlowFast, TSN, and TSM. Finally, an embedded automotive solution was used to demonstrate that both methods are running in real-time.

show abstract

Section: Related Workmentioning

confidence: 99%

Fusion Object Detection and Action Recognition to Predict Violent Action

Rodrigues

Costa

Melo

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…To evaluate the system generated depth frames and corresponding 2D ground-truth, the Part Affinity Fields [6] method was used. From it, a custom CNN was implemented consisting only on the first stage of the original PAF CNN [33], following the same training procedures. In each sub-evaluation, M#, the method used the depth frame as input features and the 2D body pose as output labels (Fig.…”

Section: D Pose Estimation From Depth Images (Ev1)mentioning

confidence: 99%

A system for the generation of in-car human body pose datasets

Borges

Queirós

Oliveira

et al. 2020

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

With the advent of autonomous vehicles, detection of the occupants' posture is crucial to tackle the needs of infotainment interaction or passive safety systems. Generative approaches have been recently proposed for human body pose in-car detection, but this type of approaches requires a large training dataset for a feasible accuracy. This requirement poses a difficulty, given the substantial time required to annotate such large amount of data. In the in-car scenario, this requirement risk increases even further, since a robust human body pose ground-truth system capable of working in it is needed but inexistent. Currently, the gold standard for human body pose capture is based on optical systems, requiring up to 39 visible markers for a plug-in gait model, which in this case are not feasible given the occlusions inside the car. Other solutions, such as inertial suits, also have limitations linked to magnetic sensitivity and global positioning drift. In this paper, a system for the generation of images for human body pose detection in an in-car environment is proposed. To this end, we propose to smartly combine inertial and optical systems to suppress their individual limitations: By combining the global positioning of 3 visible head markers provided by the optical system with the inertial suit's relative human body pose, we obtain an occlusion-ready, drift-free full-body global positioning system. This system is then spatially and temporally calibrated with a time-of-flight sensor, automatically obtaining in-car image data with (multi-person) pose annotations. Besides quantifying the inertial suit inherent sensitivity and accuracy, the feasibility of the overall system for human body pose capture in the in-car scenario was demonstrated. Our results quantify the errors associated with the inertial suit, pinpoint some sources of the system's uncertainty and propose how to minimize some of them. Finally, we demonstrate the feasibility of using system generated data (which was made publicly available), independently or mixed with two publicly available generic datasets (not in-car), to train 2 machine learning algorithms, demonstrating the improvement in their algorithmic accuracy for the in-car scenario.

show abstract

“…With this new trend, occupant monitoring through human body pose detection gains increased importance in AD. In the last decade, multiple machine learning approaches have been proposed in the literature for human body pose detection in RGB and depth images, showing high inference accuracy with low computation cost (Shotton et al, 2013;Rodrigues et al, 2019;Torres et al, 2019). However, this type of approach requires a large and generic image database for training.…”

Section: Introductionmentioning

confidence: 99%

Automated Generation of Synthetic in-Car Dataset for Human Body Pose Detection

Borges

Oliveira

Torres

et al. 2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

Self Cite

View full text Add to dashboard Cite

In this paper, a toolchain for the generation of realistic synthetic images for human body pose detection in an in-car environment is proposed. The toolchain creates a customized synthetic environment, comprising human models, car, and camera. Poses are automatically generated for each human, taking into account a per-joint axis Gaussian distribution, constrained by anthropometric and range of motion measurements. Scene validation is done through collision detection. Rendering is focused on vision data, supporting time-of-flight (ToF) and RGB cameras, generating synthetic images from these sensors. Ground-truth data is then generated, comprising the car occupants' body pose (2D/3D), as well as full body RGB segmentation frames with different body parts' labels. We demonstrate the feasibility of using synthetic data, combined with real data, to train distinct machine learning agorithms, demonstrating the improvement in their algorithmic accuracy for the in-car scenario.

show abstract

Real-Time Human Body Pose Estimation for In-Car Depth Images

Cited by 6 publications

References 21 publications

Fusion Object Detection and Action Recognition to Predict Violent Action

Fusion Object Detection and Action Recognition to Predict Violent Action

A system for the generation of in-car human body pose datasets

Automated Generation of Synthetic in-Car Dataset for Human Body Pose Detection

Contact Info

Product

Resources

About