Real Time Object Detection, Tracking, and Distance and Motion Estimation based on Deep Learning: Application to Smart Mobility

Abstract

In this paper, we will introduce our object detection, localization and tracking system for smart mobility applications like traffic road and railway environment. Firstly, an object detection and tracking approach was firstly carried out within two deep learning approaches: You Only Look Once (YOLO) V3 and Single Shot Detector (SSD). A comparison between the two methods allows us to identify their applicability in the traffic environment. Both the performances in road and in railway environments were evaluated. Secondly, object distance estimation based on Monodepth algorithm was developed. This model is trained on stereo images dataset but its inference uses monocular images. As the output data, we have a disparity map that we combine with the output of object detection. To validate our approach, we have tested two models with different backbones including VGG and ResNet used with two datasets : Cityscape and KITTI. As the last step of our approach, we have developed a new method-based SSD to analyse the behavior of pedestrian and vehicle by tracking their movements even in case of no detection on some images of a sequence. We have developed an algorithm based on the coordinates of the output bounding boxes of the SSD algorithm. The objective is to determine if the trajectory of a pedestrian or vehicle can lead to a dangerous situations. The whole of development is tested in real vehicle traffic conditions in Rouen city center, and with videos taken by embedded cameras along the Rouen tramway.

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

“…We propose here to exploit the maturity of such methods in order to enhance the perception and the analysis of a niche area of application and road scene understanding. In this context, several works related to object detection, recognition and tracking have already been carried out in collaboration with SEGULA technologies, such as the tracking of a person by a drone [3], the detection and tracking of objects for the road smart mobility [4]. The goal is to propose new approaches of embedded vision dedicated to the detection of objects, their localisation as well as their tracking, which allow better performance in realistic and complex conditions.…”

Subjective analysis of social distance monitoring using YOLO v3 architecture and crowd tracking system

“…in the company of two thousand photo of people. The amount of the classes doesn't notably effect the velocity of the inference[12]. The model of 1 class reported a 0.4 FPS more fast from the model of 80 classes.…”

“…Recently in these years, different ways lying on Convolutional Neural Networks (CNN) as backbone got recommended for dealing with the issue, which was a benchmark. These methods are split to two groups: one stage method, that maintain coordinate detection also class prediction in one step, and two stage method, that initially identify areas of the photos that object can be existing, afterward put these area to an classifier of images [12].…”

“…However, the quantity of parameters directly impacts the inference time. The fully connected layers amount parameters are directly effected by the classes suppression on the network, which although is a small part of the whole model [12].…”

Electronic Orientation Aid for Visually Impaired using Graphics Processing Unit (GPU)

“…Therefore, we build a system called EOA that will help them to carry out their activities like normal people to some extent. Various algorithms and measurements were studied from different papers like "Object Detection Based on YOLO Network" [2], "Real Time Object Detection, Tracking, and Distance and Motion Estimation based on Deep Learning: Application to Smart Mobility" [19] etc. that encouraged us to select the correct algorithm to carry out our work.…”