One For All: A Mutual Enhancement Method for Object Detection and Semantic Segmentation

Zhang, Shichao; Zhang, Zhe; Sun, Libo; Qin, Wenhu

doi:10.3390/app10010013

Cited by 18 publications

(6 citation statements)

References 38 publications

(85 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the end, a SoC design has been successfully demonstrated on ZCU102 development kit, which achieves a speeds up of the processing time saving by a factor of 2.6 comparing to its GPU implementation. [4] 360 × 720 TITAN XP 96.30% 160 ms SSLGAN [9] 375 × 1242 TITAN X 95.53% 700 ms RBNet [5] 300 × 900 Tesla K20c 94.97% 180 ms StixelNet-II [10] 800 × 370 Quadro M6000 94.88% 1200 ms MultiNet [11] 1248 × 384 94.88% 170 ms RoadNet3 [15] 600 × 160 × 5 GTX 950M 94.44% 300 ms DEEP-DIG [13] Titan X 93.98% 140 ms Up-Conv-Poly [12] 500 × 500 TITAN X 93.83% 83 ms OFA-Net [49] 93.74% 40 ms Up-Conv [12] 300 × 300 GTX TITAN X 92.39% 52.2 ms ALO-AVG-MM [50] 624 × 192 GTX 1080 92.03% 29.6 ms FTP [14] 91.61% 280 ms PT-ResNet [51] GTX 1080 Ti 91.61% 300 ms FCN-LC [48] 621 × 187 TITAN X 90.79% 30 ms StixelNet [52] 24 × 370 89.12% 1000 ms MAP [14] 87.80% 280 ms SPRAY [53] 800 × 600 GTX 580 87.09% 45 ms multi-task CNN [54] 375 × 1242 unknown type GPU 86.81% 25.1 ms PGM-ARS [55] ∼ 75 × 248 Intel i7-4700MQ processor 85.69% 50 ms SRF [56] 500 × 250 82.44% 200 ms ARSL-AMI [57] 80.36% 50 ms CN [58] 79…”

Section: Discussionmentioning

confidence: 99%

RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation

Bai

Lyu

Huang

2020

Preprint

View full text Add to dashboard Cite

In recent years, convolutional neural network has gained popularity in many engineering applications especially for computer vision. In order to achieve better performance, often more complex structures and advanced operations are incorporated into the neural networks, which results very long inference time. For time-critical tasks such as autonomous driving and virtual reality, real-time processing is fundamental. In order to reach real-time process speed, a light-weight, high-throughput CNN architecture namely RoadNet-RT is proposed for road segmentation in this paper. It achieves 90.33% MaxF score on test set of KITTI road segmentation task and 8 ms per frame when running on GTX 1080 GPU. Comparing to the state-of-the-art network, RoadNet-RT speeds up the inference time by a factor of 20 at the cost of only 6.2% accuracy loss. For hardware design optimization, several techniques such as depthwise separable convolution and non-uniformed kernel size convolution are customized designed to further reduce the processing time. The proposed CNN architecture has been successfully implemented on an FPGA ZCU102 MPSoC platform that achieves the computation capability of 83.05 GOPS. The system throughput reaches 327.9 frames per second with image size 1216×176.

show abstract

Section: Discussionmentioning

confidence: 99%

RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation

Bai

Lyu

Huang

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Road Detection In recent years, more and more researches on autonomous driving at home and abroad have accumulated a certain research foundation. OFA-Net [14] used a strategy called "1-N Alternation" to train the model, which can make a fusion of features from detection and segmentation data. RoadNet-RT [15] speeded up the inference time by optimizing Depthwise separable convolution and non-uniformed kernel size convolution.…”

Section: Related Workmentioning

confidence: 99%

A Road Segmentation Model Based on Mixture of the Convolutional Neural Network and the Transformer Network

Xu¹,

Zhao²,

Hu³

et al. 2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

Convolutional neural networks (CNN) based on U-shaped structures and skip connections play a pivotal role in various image segmentation tasks. Recently, Transformer starts to lead new trends in the image segmentation task. Transformer layer can construct the relationship between all pixels, and the two parties can complement each other well. On the basis of these characteristics, we try to combine Transformer pipeline and convolutional neural network pipeline to gain the advantages of both. The image is put into the U-shaped encoder-decoder architecture based on empirical combination of self-attention and convolution, in which skip connections are utilized for localglobal semantic feature learning. At the same time, the image is also put into the convolutional neural network architecture. The final segmentation result will be formed by Mix block which combines both. The mixture model of the convolutional neural network and the Transformer network for road segmentation (MCTNet) can achieve effective segmentation results on KITTI dataset and Unstructured Road Scene (URS) dataset built by ourselves. Codes, self-built datasets and trainable models will be available on https://github.com/xflxfl1992/MCTNet.

show abstract

“…Some SOTA algorithms were compared to the study's proposed method. Here, RBANet [29], OFANet [32], and HA-DeepLab [10] used a single sensor and camera. Likewise, ChipNet [31] and LoDNN [16] only used LiDAR in their proposed system.…”

Section: Kitti Road Benchmarkmentioning

confidence: 99%

Free Space Detection Using Camera-LiDAR Fusion in a Bird’s Eye View Plane

Lee

et al. 2021

Sensors

View full text Add to dashboard Cite

Although numerous road segmentation studies have utilized vision data, obtaining robust classification is still challenging due to vision sensor noise and target object deformation. Long-distance images are still problematic because of blur and low resolution, and these features make distinguishing roads from objects difficult. This study utilizes light detection and ranging (LiDAR), which generates information that camera images lack, such as distance, height, and intensity, as a reliable supplement to address this problem. In contrast to conventional approaches, additional domain transformation to a bird’s eye view space is executed to obtain long-range data with resolutions comparable to those of short-range data. This study proposes a convolutional neural network architecture that processes data transformed to a bird’s eye view plane. The network’s pathways are split into two parts to resolve calibration errors in the transformed image and point cloud. The network, which has modules that operate sequentially at various scaled dilated convolution rates, is designed to quickly and accurately handle a wide range of data. Comprehensive empirical studies using the Karlsruhe Institute of Technology and Toyota Technological Institute’s (KITTI’s) road detection benchmarks demonstrate that this study’s approach takes advantage of camera and LiDAR information, achieving robust road detection with short runtimes. Our result ranks 22nd in the KITTI’s leaderboard and shows real-time performance.

show abstract

One For All: A Mutual Enhancement Method for Object Detection and Semantic Segmentation

Cited by 18 publications

References 38 publications

RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation

RoadNet-RT: High Throughput CNN Architecture and SoC Design for Real-Time Road Segmentation

A Road Segmentation Model Based on Mixture of the Convolutional Neural Network and the Transformer Network

Free Space Detection Using Camera-LiDAR Fusion in a Bird’s Eye View Plane

Contact Info

Product

Resources

About