Mixed-Scale Unet Based on Dense Atrous Pyramid for Monocular Depth Estimation

Abstract

Monocular depth estimation is an undirected problem, so constructing a network to predict better image depth information is an important research topic. This paper proposes a mixed-scale Unet network (MAPNet) with a dense atrous pyramid based on the coder-decoder structure widely used in computer vision. We innovatively introduce the Unet++ structure of the image segmentation network for depth estimation. We reset the number of convolutional layers of the network under the framework of the Unet++ network and innovatively connect the decoders densely. Moreover, by choosing the appropriate size of the atrous radius, we form a dense atrous pyramid based on different feature layers to better connect the features in the deep and shallow layers of the network. To verify the effectiveness of the proposed algorithm, we tested the network on the KITTI public dataset and the NYU Depth V2 dataset. We compared it with the current state-of-the-art methods. The proposed method has higher accuracy and has steadily improved relative to the threshold of accuracy and root-mean-square error. We also conduct ablation studies, studies targeting the effectiveness of the network framework, and discussions on the convergence time and parameter complexity of the network. We will open-source the code at https://github.com/yang-yi-fan/MAPUnet.

Improving Speech Enhancement Framework via Deep Learning

“…This module contains four high-performance digital microphones (MP34DT01-M), built-in capacitive sensing elements, and an I2C interface, supports far-field voice capture, and far-field voice capture recording, and understands needs within a range of up to 5 m. Figs. 5a and 5b present the diagrams of the ReSpeaker Mic Array v2.0.1 and the module system [19], respectively. The module also improves recording quality, reduces ambient voice echo, and employs AEC to eliminate current audio output.…”

Hybrid Skip: A Biologically Inspired Skip Connection for the UNet Architecture

“…Apart from an improvement in dense and boundary segmentation, the residual skip connections also exhibited robustness to noise. In a similar fashion, MAPUNet [58], inspired by UNet++ [66], and UNet 3+ [19], exploited multi-scale feature fusion and supervision for monocular depth estimation. Moreover, a UNet++ variant with residual blocks and dense gated convolution based attention [60] was used for monocular depth estimation using sparse depth measurements [64].…”

Comparing U-Net based architectures in monocular depth estimation

“…Devido a sua arquitetura flexível e facilmente modificável, a U-Net tem sido uma arquitetura de CNN comumente adotada como base para a construc ¸ão de novas redes na tarefa de estimativa de profundidade [29,30,31]. Saxena et al [31] combinaram a U-Net, utilizando a EfficientNet como codificador, com a reconstruc ¸ão de mapas de profundidades ruidosos para melhorar significativamente a estimativa de profundidade.…”

“…Jan and Seo [32] substituíram camadas simples de convoluc ¸ão 2D da U-Net por camadas de convoluc ¸ões residuais para aprimorar a extrac ¸ão de características mais expressivas, além de adicionarem um mecanismo de atenc ¸ão antes de cada camada do decodificador, permitindo que a rede também seja capaz de identificar pequenas características e de construir mapas de profundidade refinados. Yang et al [29] propuseram uma arquitetura supervisionada baseada na UNet++ (versão da U-Net que incluiu conexões aninhadas entre codificador e decodificador [14]) e adotaram pirâmides de convoluc ¸ões dilatadas para reduzir o custo computacional e ampliar a capacidade da rede para capturar características da imagem em diversas escalas. Com o intuito de desenvolver uma soluc ¸ão mais leve, Guzzo and Gazolli [33] propuseram uma abordagem para estimativa de profundidade que utiliza a arquitetura UNet++ empregando como codificador a rede neural MobileNetV2 pré-treinada.…”