Decouple the High-Frequency and Low-Frequency Information of Images for Semantic Segmentation

Abstract

As a special kind of signal processing technology, image processing has been developed rapidly after the appearance of convolutional neural network (CNN). At present, the semantic segmentation methods are all based on CNN and ignore the advantages of traditional image processing technology. We combine the two and make them promote each other. The high frequency component of the image represents the edge part and the low frequency represents the body part. Based on this assumption, we use Fourier transform to obtain the high and low frequency component from images. Then, a multibranch parallel network structure is designed, and the high and low frequency components are sent into two branches respectively to obtain the body and edge features. Finally, the two features are fused together through one deep feature fusion to obtain the final output. In this way, the edge information and body information are decoupled on original images and extracted separately, which not only ensures the consistency of the internal information within objects, but also strengthens the supervision of the edge part which is the most error prone area in semantic segmentation. The results on Cityscapes and KITTI fully demonstrate the effectiveness of our work.

Fast real-time semantic segmentation for autonomous driving

“…Instead of a fully connected layer, Different from the traditional image block-based segmentation methods, FCN proves that the end-to-end, pixel-to-pixel training mode of a convolutional neural network can significantly improve the computational efficiency and prediction performance of semantic segmentation and the end-to-end training paves the way for the development of subsequent semantic segmentation algorithms. FCN ADAPTS all the fully connected layers of the traditional convolutional network into dense convolutional layers of the corresponding size; for example, on the basis of VGGNet, FCN ADAPTS [8][9][10][11][12] the last three layers of the VGGNet network into multi-channel convolutional layers with the same vector length corresponding to the 1×1 convolution kernel.…”

Facial expression recognition combining different resolution images

“…There are also two problems with the neural network-based approach. First, simple neural networks (such as FNN) [6][7][8][9][10][11] ignore the two-dimensional information of the image. Second, the features extracted by shallow convolutional networks have poor robustness.…”

Improve the speed of semantic segmentation models from multiple perspectives

“…in each pixel of the satellite image, land cover classification can be viewed as a multi-level semantic segmentation task [1][2][3][4][5][6]. Road and building detection is also an important research topic of traffic management, urban planning [8] and road monitoring [4]. Autonomous driving is a complex robotic task that requires perception, planning and execution in an ever-changing environment.…”