People Identification in Private Car Using 3D LiDAR With Generative Image Inpainting and YOLOv5

Abstract

People Identification is a critical aspect in developing modern vehicles, aimed at enhancing safety and comfort levels. Most traditional methods of people identification in vehicles use RGB images or videos. In this study, we introduce a novel methodology for identifying individuals in private car scenarios, utilizing 3D Light Detection and Ranging (LiDAR) technology, generative image inpainting based on Contextual Attention, and the YOLOv5 model. Initially, we gather data utilizing a 3D-LiDAR instrument and subsequently convert the acquired depth data into depth images. Following this, the depth images are annotated manually to indicate the positions and identifiers of various individuals occupying distinct seats. This annotated data serves as the training material for the YOLOv5 model, facilitating the recognition and categorization of subjects. However, given that individuals seated in the back often have parts of their bodies occluded by the front seats and the passengers in them, we employ generative image inpainting techniques to reveal the occluded portions. This step significantly enhances the precision in detecting and identifying individuals situated in the back seats. We implemented our strategy on a restricted group of four participants, conducting training and testing phases within identical environments. Prior to the inpainting process, the classification's F1 score stood at 66.5%. After inpainting, we observed a notable surge in the F1 score for the rear-seat passengers increased by 17.1%.