Deep Feature-Level Sensor Fusion Using Skip Connections for Real-Time Object Detection in Autonomous Driving

John, Vijay; Mita, Seiichi

doi:10.3390/electronics10040424

Cited by 25 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…eir experiments proved that the model could reduce computational overhead effectively. John and Mita [24] proposed a residual semantic-guided attention feature pyramid network, including input and output branches. e model used the input branch to extract the features of a single sensor and then used the residual connection to integrate the extracted features into the output perception branch.…”

Section: One-stage Object Detection Modelmentioning

confidence: 99%

Earf-YOLO: An Efficient Attention Receptive Field Model for Recognizing Symbols of Zhuang Minority Patterns

Wang

Yan

Qin

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

As for recognizing Zhuang minority pattern symbols, current recognition models often cause high computational overhead and low accuracy since Zhuang minority pattern symbols have large feature vectors and some complex features. In this paper, we present the efficient attention receptive field you only look once (Earf-YOLO), a new scheme to address those problems. Firstly, a global-local-transformer (GLocalT) structure is proposed, through which other control systems are introduced into the axial self-attention module, and global-local training strategies are also designed. The structure can use other control systems to compensate for the lost feature information along the height, width, and channel axes. The global-local training strategy can encode long-term dependencies between features and reduce local information loss, fully illustrating that the structure has high feature expression ability. Besides, strength receptive field block (SRFB) is suggested to use the dilated convolution to control the receptive field’s eccentricity and enrich the feature information of the receptive field during its training. With more branches, it can better extract multiscale features, enrich the feature space of the convolution block, and reparametrize multibranch during prediction to fuse them into the main branch, all of which contribute to the improvement of the model performance. Finally, some advanced training techniques are adopted to enhance the detection effect further. In the end, comparative experiments are conducted on the datasets of Zhuang pattern symbols and PASCAL VOC, whose results indicate that the AP and FPS of the suggested model reach their highest values, manifesting its high efficiency.

show abstract

Section: One-stage Object Detection Modelmentioning

confidence: 99%

Earf-YOLO: An Efficient Attention Receptive Field Model for Recognizing Symbols of Zhuang Minority Patterns

Wang

Yan

Qin

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Monocular cameras tend to have a longer range compared to stereo cameras. Thermal/infrared cameras are also used to detect objects in low-lighting conditions (e.g., Korthals et al, 2018;John and Mita, 2021). The field of view depends on the focal length of the lens used.…”

Section: Performance Metrics For Environment Perceptionmentioning

confidence: 99%

A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

Sharath

Mehran

2021

Front. Future Transp.

View full text Add to dashboard Cite

The article presents a review of recent literature on the performance metrics of Automated Driving Systems (ADS). More specifically, performance indicators of environment perception and motion planning modules are reviewed as they are the most complicated ADS modules. The need for the incorporation of the level of threat an obstacle poses in the performance metrics is described. A methodology to quantify the level of threat of an obstacle is presented in this regard. The approach involves simultaneously considering multiple stimulus parameters (that elicit responses from drivers), thereby not ignoring multivariate interactions. Human-likeness of ADS is a desirable characteristic as ADS share road infrastructure with humans. The described method can be used to develop human-like perception and motion planning modules of ADS. In this regard, performance metrics capable of quantifying human-likeness of ADS are also presented. A comparison of different performance metrics is then summarized. ADS operators have an obligation to report any incident (crash/disengagement) to safety regulating authorities. However, precrash events/states are not being reported. The need for the collection of the precrash scenario is described. A desirable modification to the data reporting/collecting is suggested as a framework. The framework describes the precrash sequences to be reported along with the possible ways of utilizing such a valuable dataset (by the safety regulating authorities) to comprehensively assess (and consequently improve) the safety of ADS. The framework proposes to collect and maintain a repository of precrash sequences. Such a repository can be used to 1) comprehensively learn and model the precrash scenarios, 2) learn the characteristics of precrash scenarios and eventually anticipate them, 3) assess the appropriateness of the different performance metrics in precrash scenarios, 4) synthesize a diverse dataset of precrash scenarios, 5) identify the ideal configuration of sensors and algorithms to enhance safety, and 6) monitor the performance of perception and motion planning modules.

show abstract

“…Although this method can improve the problem of Yang's work [15], it is still difficult to obtain the accurate velocity when the obstacle is occluded or the shape changes dramatically. Feature level fusion mainly extracts features of different sensors for fusion [26], which is mostly used for heterogeneous sensors. Its processed data between data level fusion and decision level fusion, and data loss is not serious.…”

Section: Related Workmentioning

confidence: 99%

Obstacle detection and tracking algorithm based on multi‐lidar fusion in urban environment

Liu²,

Zhang

et al. 2021

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Adjacent obstacles are difficult to be distinguished, and remote obstacle is easily detected as multiple targets. Besides, occluded obstacles are difficult to track, and tracking velocity of the obstacles in rapid acceleration or deceleration converges slowly in the urban environment. In view of this, an obstacle detection and tracking method based on multi-lidar is proposed. Firstly, based on the vehicle kinematics model, motion compensation is adopted to solve the space-time synchronization problem among lidars after road segmentation, and data level fusion is completed. Next, the obstacles are detected by combining adaptive voxel grid DBSCAN (AVG-DBSCAN) algorithm and Region Growing (RG) algorithm. Then, the Breadth First Search (BFS) algorithm and KD tree are adopted to improve the Evolutionary Hungarian algorithm for the fast association of obstacles. Finally, the occlusion rate and dynamic region selection are used to track obstacles accurately, based on the Kalman Filter of uniform acceleration model. The experimental results on the authors' extracted urban dataset show that the proposed method can effectively fuse multi-lidar data and outperforms other methods in obstacle detection and tracking. Its average accuracy for detection is 97.53% and its average accuracy for tracking is 95.1%. The average duration of the entire process is only 30 ms.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Deep Feature-Level Sensor Fusion Using Skip Connections for Real-Time Object Detection in Autonomous Driving

Cited by 25 publications

References 28 publications

Earf-YOLO: An Efficient Attention Receptive Field Model for Recognizing Symbols of Zhuang Minority Patterns

Earf-YOLO: An Efficient Attention Receptive Field Model for Recognizing Symbols of Zhuang Minority Patterns

A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

Obstacle detection and tracking algorithm based on multi‐lidar fusion in urban environment

Contact Info

Product

Resources

About