Road segmentation with image-LiDAR data fusion in deep neural network

Liu, Huafeng; Yao, Yazhou; Sun, Zeren; Li, Xiangrui; Ke, Jia; Tang, Zhenmin

doi:10.1007/s11042-019-07870-0

Cited by 15 publications

(9 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many scholars have carried out useful research on image segmentation [16][17][18][19][20][21][22][23][24][25][26][27]. Zhao et al [16] into the ACOR and improved the selection mechanism of the original ACOR to form an improved algorithm (CCACO) for the first time.…”

Section: Related Workmentioning

confidence: 99%

“…Zhao et al [16] into the ACOR and improved the selection mechanism of the original ACOR to form an improved algorithm (CCACO) for the first time. Liu et al [17] proposed a novel structure to fuse image and LiDAR point cloud in an end-toend semantic segmentation network, in which the fusion is performed at the decoder stage instead of at, more commonly, the encoder stage. Ji et al [18] proposed a new architecture of feature aggregation, which is designed to deal with the problem that the information of each convolutional layer cannot be used reasonably and the shallow layer information is lost in the process of transmission.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

The Segmentation of Road Scenes Based on Improved ESPNet Model

Jin

Han

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

Image segmentation is an important research in image processing and machine vision in which automated driving can be seen the main application scene of image segmentation algorithms. Due to the many constraints of power supply and communication in in-vehicle systems, the vast majority of current image segmentation algorithms are implemented based on the deep learning model. Despite the ultrahigh segmentation accuracy, the problem of mesh artifacts and segmentation being too severe is obvious, and the high cost, computational, and power consumption devices required are difficult to apply in real-world scenarios. It is the focus of this paper to construct a road scene segmentation model with simple structure and no need of large computing power under the premise of certain accuracy. In this paper, the ESPNet (Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation) model is introduced in detail. On this basis, an improved ESPNet model is proposed based on ESPNet. Firstly, the network structure of the ESPNet model is optimized, and then, the model is optimized by using a small amount of weakly labeled and unlabeled scene sample data. Finally, the new model is applied to video image segmentation based on dash cam. It is verified on Cityscape, PASCAL VOC 2012, and other datasets that the algorithm proposed in this paper is faster, and the amount of parameters required is less than 1% of other algorithms, so it is suitable for mobile terminals.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

The Segmentation of Road Scenes Based on Improved ESPNet Model

Jin

Han

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Through the complementary process between vision and LiDAR (Morales et al, 2021;Mutz et al, 2021), the performance of adaptive cruise control was significantly improved; thus, a complementary method combining vision and LiDAR was developed in order to further improve the accuracy of unmanned aerial vehicle (UAV) navigation (Yu et al, 2021). Liu et al proposed a new structure of LiDAR supplement vision in an end-to-end semantic segmentation network, which can effectively improve the performance of automatic driving (Liu et al, 2020). The above methods had good application effects in the field of autonomous driving (Chen et al, 2021;Yang et al, 2021;Zhang et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Inter-row information recognition of maize in the middle and late stages via LiDAR supplementary vision

Xie

Liu

et al. 2022

Front. Plant Sci.

View full text Add to dashboard Cite

In the middle and late stages of maize, light is limited and non-maize obstacles exist. When a plant protection robot uses the traditional visual navigation method to obtain navigation information, some information will be missing. Therefore, this paper proposed a method using LiDAR (laser imaging, detection and ranging) point cloud data to supplement machine vision data for recognizing inter-row information in the middle and late stages of maize. Firstly, we improved the YOLOv5 (You Only Look Once, version 5) algorithm based on the characteristics of the actual maize inter-row environment in the middle and late stages by introducing MobileNetv2 and ECANet. Compared with that of YOLOv5, the frame rate of the improved YOLOv5 (Im-YOLOv5) increased by 17.91% and the weight size decreased by 55.56% when the average accuracy was reduced by only 0.35%, improving the detection performance and shortening the time of model reasoning. Secondly, we identified obstacles (such as stones and clods) between the rows using the LiDAR point cloud data to obtain auxiliary navigation information. Thirdly, the auxiliary navigation information was used to supplement the visual information, so that not only the recognition accuracy of the inter-row navigation information in the middle and late stages of maize was improved but also the basis of the stable and efficient operation of the inter-row plant protection robot was provided for these stages. The experimental results from a data acquisition robot equipped with a camera and a LiDAR sensor are presented to show the efficacy and remarkable performance of the proposed method.

show abstract

“…In addition, the increasing interest in neural networks and deep learning stimulates new ideas for image segmentation and classification [25][26][27][28]. Luca Caltagirone et al used a fully convolutional neural network for light detection and ranging lidar fusion to improve the reliability of unstructured environment recognition results [29].…”

Section: Introductionmentioning

confidence: 99%

Detection and Modeling of Unstructured Roads in Forest Areas Based on Visual-2D Lidar Data Fusion

Lei

Yao

Zhao

et al. 2021

Forests

View full text Add to dashboard Cite

The detection and recognition of unstructured roads in forest environments are critical for smart forestry technology. Forest roads lack effective reference objects and manual signs and have high degrees of nonlinearity and uncertainty, which pose severe challenges to forest engineering vehicles. This research aims to improve the automation and intelligence of forestry engineering and proposes an unstructured road detection and recognition method based on a combination of image processing and 2D lidar detection. This method uses the “improved SEEDS + Support Vector Machine (SVM)” strategy to quickly classify and recognize the road area in the image. Combined with the remapping of 2D lidar point cloud data on the image, the actual navigation requirements of forest unmanned navigation vehicles were fully considered, and road model construction based on the vehicle coordinate system was achieved. The algorithm was transplanted to a self-built intelligent navigation platform to verify its feasibility and effectiveness. The experimental results show that under low-speed conditions, the system can meet the real-time requirements of processing data at an average of 10 frames/s. For the centerline of the road model, the matching error between the image and lidar is no more than 0.119 m. The algorithm can provide effective support for the identification of unstructured roads in forest areas. This technology has important application value for forestry engineering vehicles in autonomous inspection and spraying, nursery stock harvesting, skidding, and transportation.

show abstract

Road segmentation with image-LiDAR data fusion in deep neural network

Cited by 15 publications

References 43 publications

The Segmentation of Road Scenes Based on Improved ESPNet Model

The Segmentation of Road Scenes Based on Improved ESPNet Model

Inter-row information recognition of maize in the middle and late stages via LiDAR supplementary vision

Detection and Modeling of Unstructured Roads in Forest Areas Based on Visual-2D Lidar Data Fusion

Contact Info

Product

Resources

About