A deep learning based image enhancement approach for autonomous driving at night

Li, Guofa; Yang, Yifan; Qu, Xingda; Cao, Dongpu; Li, Keqiang

doi:10.1016/j.knosys.2020.106617

Cited by 160 publications

(60 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These advancements have not only been in the areas of physiological sensing and wearable devices. Over the past decade, there have been significant improvements in computer vision and machine learning approaches, where we can now accurately detect specific features and behaviors of drivers from the in-cabin videos while detecting objects [36] and outside conditions through the outdoor videos [37]. However, since the majority of existing NDS were introduced over a decade ago, many of the existing datasets do not include these modalities of data such as driver's pose features, gaze patterns, and objects in the environment.…”

Section: Introductionmentioning

confidence: 99%

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

et al. 2021

View full text Add to dashboard Cite

Effective shared autonomy requires a clear understanding of driver's behavior, which is governed by multiple psychophysiological and environmental variables. Disentangling this intricate web of interactions requires understanding the driver's state and behaviors in different real-world scenarios, longitudinally. Naturalistic Driving Studies (NDS) have shown to be an effective approach to understanding the driver's state and behavior in real-world scenarios. However, due to the lack of technological and computing capabilities, former NDS only focused on vision-based approaches, ignoring important psychophysiological factors such as cognition and emotion. The main objective of this paper is to introduce HARMONY, a human-centered multimodal naturalistic driving study, where driver's behaviors and states are monitored through (1) in-cabin and outside video streams (2) physiological signals including driver's heart rate and hand acceleration (IMU data), (3) ambient noise, light, and the vehicle's GPS location, and (4) music logs, including song features such as tempo. HARMONY is the first study that collects long-term naturalistic facial, physiological, and environmental data simultaneously. This paper summarizes HARMONY's goals, framework design, data collection and analysis, and the on-going and future research efforts. Through a presented case study, we first demonstrate the importance of longitudinal driver state sensing through using Kernel Density Estimation Methods. Second, we leverage the application of Bayesian Change Point detection methods to demonstrate how we can identify driver behaviors and responses to the environmental conditions by fusing psychophysiological information with features extracted from video streams.INDEX TERMS Naturalistic driving study, physiological sensing,driver state detection, shared-autonomy, contextual awareness, human-in-the-loop systems

show abstract

Section: Introductionmentioning

confidence: 99%

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, the proposed methods are based on insulator images captured in daytime, while detecting insulator defects at night is still a challenging task. Therefore, one of our future work will focus on improving the night detection performance by using image enhancement technologies [51]. Moreover, researchers have introduced the concept of Corner Net [52] and Extreme Net [53] into target detection in the recent years, and satisfactory results have been reported.…”

Section: Discussionmentioning

confidence: 99%

Deep Learning Approaches on Defect Detection in High Resolution Aerial Images of Insulators

Wen

Luo

Chen

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

By detecting the defect location in high-resolution insulator images collected by unmanned aerial vehicle (UAV) in various environments, the occurrence of power failure can be timely detected and the caused economic loss can be reduced. However, the accuracies of existing detection methods are greatly limited by the complex background interference and small target detection. To solve this problem, two deep learning methods based on Faster R-CNN (faster region-based convolutional neural network) are proposed in this paper, namely Exact R-CNN (exact region-based convolutional neural network) and CME-CNN (cascade the mask extraction and exact region-based convolutional neural network). Firstly, we proposed an Exact R-CNN based on a series of advanced techniques including FPN (feature pyramid network), cascade regression, and GIoU (generalized intersection over union). RoI Align (region of interest align) is introduced to replace RoI pooling (region of interest pooling) to address the misalignment problem, and the depthwise separable convolution and linear bottleneck are introduced to reduce the computational burden. Secondly, a new pipeline is innovatively proposed to improve the performance of insulator defect detection, namely CME-CNN. In our proposed CME-CNN, an insulator mask image is firstly generated to eliminate the complex background by using an encoder-decoder mask extraction network, and then the Exact R-CNN is used to detect the insulator defects. The experimental results show that our proposed method can effectively detect insulator defects, and its accuracy is better than the examined mainstream target detection algorithms.

show abstract

“…More specifically, the positive detection of a target can always trigger the tracking of a specific object. In other words, the proposed approach may not work well at night or some cases of insufficient lighting(hazy weather) [49]- [51]. Second, the proposed approach effectively fits the 3D vehicle model into the image frame when the vehicle is tracked, but the vehicle model fitting evolutionary procedure is time-consuming.…”

Section: E Discussionmentioning

confidence: 99%

Deformable Model-Based Vehicle Tracking and Recognition Using 3-D Constrained Multiple-Kernels and Kalman Filter

Liu

2021

IEEE Access

View full text Add to dashboard Cite

Video-based vehicle tracking and recognition is an important application in the Intelligent Transportation System. High similarity among vehicle types, frequent occlusion and low video resolution in traffic surveillance are the major problems in this research area. In this paper, we proposed a vehicle tracking system by using 3-D constrained multiple-kernels, facilitated with Kalman filtering, to continuously update the location of the moving vehicles. To further robustly and efficiently track vehicles that are partially or even fully occluded, evolutionary optimization is applied to camera calibration for systematically building 3-D vehicle model, from which we can extract the vehicle's features such as the vehicle type, color and license plate. Then, a self-similarity descriptor is further introduced for vehicle re-identification. The proposed system is evaluated on the NVIDIA AI City Datasets and one self-recorded high-resolution video. The experimental results have shown the favorable performance, which not only can successfully track vehicles under occlusion, but also can maintain the knowledge of 3-D vehicle geometry.INDEX TERMS 3-D vehicle modeling, constrained multiple kernels, camera calibration, Kalman filter, vehicle tracking and recogniton,

show abstract

A deep learning based image enhancement approach for autonomous driving at night

Cited by 160 publications

References 59 publications

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

HARMONY: A Human-Centered Multimodal Driving Study in the Wild

Deep Learning Approaches on Defect Detection in High Resolution Aerial Images of Insulators

Deformable Model-Based Vehicle Tracking and Recognition Using 3-D Constrained Multiple-Kernels and Kalman Filter

Contact Info

Product

Resources

About