Automated Generation of Synthetic in-Car Dataset for Human Body Pose Detection

Abstract

In this paper, a toolchain for the generation of realistic synthetic images for human body pose detection in an in-car environment is proposed. The toolchain creates a customized synthetic environment, comprising human models, car, and camera. Poses are automatically generated for each human, taking into account a per-joint axis Gaussian distribution, constrained by anthropometric and range of motion measurements. Scene validation is done through collision detection. Rendering is focused on vision data, supporting time-of-flight (ToF) and RGB cameras, generating synthetic images from these sensors. Ground-truth data is then generated, comprising the car occupants' body pose (2D/3D), as well as full body RGB segmentation frames with different body parts' labels. We demonstrate the feasibility of using synthetic data, combined with real data, to train distinct machine learning agorithms, demonstrating the improvement in their algorithmic accuracy for the in-car scenario.

Fusion Object Detection and Action Recognition to Predict Violent Action

“…Regarding to the works on the in-car environment, the work proposed by Torres et al [ 15 ] can detect the human body pose inside the vehicle by using a time-of-flight sensor which that could provide more light immunity compared to RGB sensors. In [ 16 ], Dixe et al use a similar approach to detect multi-person human body detection by using depth images generated synthectically with the knowledge of the work developed by Borges et al in [ 17 , 18 ]. Although, in [ 19 ], Dixe et al follow a different approach by using generative adversarial networks (GANs) for automatically generating artificial images of vehicle interiors to support the developed algorithms for the creation of monitoring systems.…”

AI based monitoring violent action detection data for in-vehicle scenarios

“…Timestamps were provided through the recording system real-time clock (RTC), while the initial time delay was extracted from each sensor number of samples in buffer divided by the frame rate. Data source location Guimarães, Portugal, University of Minho, Algoritmi Center, Latitude: 41.450715, Longitude: -8.293490 Data accessibility Repository name: datarepositorium Direct URL to InCar data: https://doi.org/10.34622/datarepositorium/1S8QVP Direct URL to InVicon data: https://doi.org/10.34622/datarepositorium/WWUTUT Related research article João Borges, Bruno Oliveira, Helena Torres, Nelson Rodrigues, Sandro Queirós, Maximilian Shiller, Victor Coelho, Johannes Pallauf, José Mendes, Jaime Fonseca, Automated Generation of Synthetic in-Car Dataset for Human Body Pose Detection [1] doi: 10.5220/0009316205500557 João Borges, Sandro Queirós, Bruno Oliveira, Helena Torres, Nelson Rodrigues, Victor Coelho, Johannes Pallauf, José Henrique Brito, José Mendes, Jaime C. Fonseca, A system for the generation of in-car human body pose datasets [2] doi: 10.1007/s00138-020-01131-z …”

“…The previous work developed on [1] consists of a toolchain for the generation of realistic synthetic for human body pose detection in an in-car environment. This toolchain demonstrated the potential for increased algorithm accuracy during body pose estimation, although the toolchain could not give the data realism that is so important for real use cases.…”