Efficient Adaptive Upsampling Module for Real-Time Semantic Segmentation

Abstract

Upsampling operation is necessary for semantic segmentation and other pixel-level prediction tasks. Among the commonly used upsampling operations, some are too simple to effectively recover the spatial details lost during downsampling process, and some are too complex and have high computation complexity. In real-world applications, it is critical to achieve high accuracy and maintain real-time inference speed. Therefore, an efficient upsampling operation is essential for these tasks. In this paper, we introduce efficient adaptive upsampling module (EAUM) for real-time semantic segmentation. Inspired by dynamic filter networks, EAUM adaptively predicts the kernel weight of each point in the upsampled feature map according to the corresponding points in the input feature map. To reduce computational cost, EAUM decomposes the spatial information and channel information required for upsampling. The proposed EAUM shows impressive performance on Cityscapes and CamVid benchmarks. Specifically, DenseENet with EAUM outperforms the baseline by 1.4% [Formula: see text] and 1.6% [Formula: see text] in accuracy with a slight drop in inference speed on Cityscapes test dataset.

Tire defect detection based on low and high-level feature fusion

“…Park and Paik [32] proposed a pyramid attention upsampling module for object detection by dividing the upsampling process into two branches to extract global contextual information and interpolate and scale feature maps, thereby reducing the loss of semantic information. Yang et al [33] proposed an efficient adaptive upsampling module for real-time semantic segmentation by adaptively predicting the weight of each piexl in the upsampling feature map based on the input feature map, which is also enlightening for object detection tasks. Luo et al [34] proposed a pixel shuffle upsampling decoder to address the problem of missing and blurry contours of small objects.…”