AI based monitoring violent action detection data for in-vehicle scenarios

Rodrigues, Nelson; Costa, Nuno M. C. da; Novais, Rita; Fonseca, Jaime C.; Cardoso, Paulo; Borges, João

doi:10.1016/j.dib.2022.108564

Cited by 4 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, BMW, Ford, Volkswagen, and Hyundai have all partnered with various companies to develop autonomous vehicles for ride-sharing and on-demand services, with production being planned in 2021 [ 8 , 9 , 10 ]. Daimler has partnered with Uber to allow for the introduction of autonomous vehicles in Uber’s ride-sharing network [ 6 ]. Toyota has also partnered with Uber with the same goal [ 11 ].…”

Section: Related Workmentioning

confidence: 99%

“…For the violent action, we used the MoLa InCar dataset [ 6 ]. This dataset was recorded with RGB, Depth, Thermal, and Event-based data (see Figure 4 ).…”

Section: Dataset Preparationmentioning

confidence: 99%

“…As such, no further customization was needed. In our experiments, we evaluate five state-of-the-art action recognition methods on the MoLa InCar dataset [ 6 ], the I3D [ 22 ], R(2+1)D [ 23 ], the Slow-Fast [ 31 ], TSN [ 32 ], and TSM [ 33 ]. We started by training our dataset in 3D convolutional networks.…”

Section: Algorithmic Analysismentioning

confidence: 99%

“…We propose an embedding object detection and violent action detection into one pipeline to achieve the best possible performance in human violent action recognition. For object detection data, we introduced the procedure of collecting and restructuring public datasets, namely, COCO [ 4 ] and TAO [ 5 ], which we fused to create the MoLa InCar dataset [ 6 ], an aggressive action and object custom dataset. In object detection, we tested different versions of the YOLO detector and selected one of the active-development SOTAs, YOLOv5 [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

“…From these studies, we were able to define the best object detection method for lost items and aggressive objects, which can be easily inserted into complex video processing pipelines for existing projects. For action recognition, we benchmark our dataset [ 6 ] end-to-end features (RGB, event frames, and optical flow,) using SOTA 2D and 3D CNN models. To conclude, we present our final solution, running all the algorithms in real-time on an embedded system.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Fusion Object Detection and Action Recognition to Predict Violent Action

Rodrigues

Costa

Melo

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In the context of Shared Autonomous Vehicles, the need to monitor the environment inside the car will be crucial. This article focuses on the application of deep learning algorithms to present a fusion monitoring solution which was three different algorithms: a violent action detection system, which recognizes violent behaviors between passengers, a violent object detection system, and a lost items detection system. Public datasets were used for object detection algorithms (COCO and TAO) to train state-of-the-art algorithms such as YOLOv5. For violent action detection, the MoLa InCar dataset was used to train on state-of-the-art algorithms such as I3D, R(2+1)D, SlowFast, TSN, and TSM. Finally, an embedded automotive solution was used to demonstrate that both methods are running in real-time.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For the violent action, we used the MoLa InCar dataset [ 6 ]. This dataset was recorded with RGB, Depth, Thermal, and Event-based data (see Figure 4 ).…”

Section: Dataset Preparationmentioning

confidence: 99%