Robust object detection under harsh autonomous‐driving environments

Abstract

In the autonomous driving environment, object instances in an image can be affected by various factors such as camera, driving state, weather, and system component. However, the deep learning-based vision systems are vulnerable to perturbation, which contains noise. Thus, robust object detection under harsh autonomous-driving environments is a more difficult than the generic situation. In this paper, it is found that not only the accuracy, but also the speed of the non-maximum suppression-based detector can be degraded under harsh environments. Therefore, object detection is handled under a harsh situation with adversarial mechanisms such as adversarial training and adversarial defence. Adversarial defence modules are designed to improve robustness in feature extraction level and define perturbations under a harsh environment for training object detectors to improve the robustness of the model's decision boundary. The proposed adversarial defence and training mechanisms improve the object detector in both accuracy and speed. The proposed method shows a 43.7% mean average precision for the COCO2015 dataset in generic object detection and 39.0% mean average precision for the BDD100K dataset in a driving environment. Furthermore, it achieves a real-time capability of 23 frames per second. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Robust Object Detection Against Multi-Type Corruption Without Catastrophic Forgetting During Adversarial Training Under Harsh Autonomous-Driving Environments

“…The second type of examples [10], [64], [65] utilize a ROD framework against corruption and perturbation. The study in [64] has the most similar goal to ours, investigating how to prevent CF in multi-domain situations.…”

“…The study in [65] uses a feature alignment method that is methodologically similar to ours; however, this study does not consider multi-type corruption, only focusing on singletype perturbations. The study in [10] has the same environment as ours, but there is a limitation that multiple object detectors are required for multi-type corruption. Therefore, ROD against multi-type corruption without CF is proposed for overcoming harsh autonomous-driving environments.…”

“…Tencent reported that in a real autonomous car, a simple attack method that marks three points on the road can cause a serious problem of reverse driving [8]. Additionally, adversarial examples [9], [10] can occur not only during malicious situations, such as adversarial attacks with perturbation, but also as non-malicious situations with corruption caused by noise, the movement of cars, weather, and digital systems. Noise can also occur in cameras or electric and electronic systems due to functional failure.…”

Robustness testing of AI systems: A case study for traffic sign recognition

“…When considering research on the robustness of AI systems to perturbations that may occur naturally during operation and should thus be much easier to cope with than malicious attacks, results are much more scarce. Some publications deal with the robustness to natural perturbations and propose measures for increasing it [17,18,16], but they do not set out to perform a fine-grained and systematic assessment of the phenomenon. Probably the work most closely related to ours is [23,22].…”

A deep grouping fusion neural network for multimedia content understanding

“…Therefore, how to accurately extract and properly fuse the semantic features of each media data for jointly semantics analysis, is the key to understand multimedia contents. Recently, Deep Neural Networks (DNNs) have shown great power in feature representation learning and achieved superior performance on various tasks, including image recognition [3][4][5], object detection [6][7][8] etc. However, they still have the following drawbacks on the multimedia content understanding, which if addressed would improve their performance.…”