Implicity: City Modeling From Satellite Images With Deep Implicit Occupancy Fields

Abstract

Abstract. High-resolution optical satellite sensors, combined with dense stereo algorithms, have made it possible to reconstruct 3D city models from space. However, these models are, in practice, rather noisy and tend to miss small geometric features that are clearly visible in the images. We argue that one reason for the limited quality may be a too early, heuristic reduction of the triangulated 3D point cloud to an explicit height field or surface mesh. To make full use of the point cloud and the underlying images, we introduce IMPLICITY, a neural representation of the 3D scene as an implicit, continuous occupancy field, driven by learned embeddings of the point cloud and a stereo pair of ortho-photos. We show that this representation enables the extraction of high-quality DSMs: with image resolution 0.5 m, IMPLICITY reaches a median height error of ≈0.7m and outperforms competing methods, especially w.r.t. building reconstruction, featuring intricate roof details, smooth surfaces, and straight, regular outlines.

Deep-FG-DSM: Fine-Grained DSM for High-resolution UAV Imagery Based on Deep Implicit Occupancy Networks

“…CON addresses these limitations by utilizing PointNet encoder to feature for each point, introducing U-Net-like (Ronneberger, et al, 2015) architecture for convolutional operations, and then Occupancy Network (Mescheder et al, 2019) is used to decode the occupancy probability of each point for 3D reconstruction. Stucker et al (2022) proposed IMPLICITY that can implicitly generate cityscale DSM, basically, it utilizes the CON network architecture to investigate large-scale 3D reconstruction together with satellite imagery. The generated DSM (Digital Surface Model) can preserve visible details from the original high-resolution UAV imagery, however, due to insufficient attention to local feature structures during the reconstruction process, and an excessive focus on buildings while neglecting non-building features, the Digital Surface Model (DSM) exhibits certain local deformations.…”

“…Inspired by the idea of IMPLICITY (Stucker et al, 2022), as Image Encoder. Given the fact that point cloud typically represents limited 3D space which is hard to retain highfrequency information where there is no point, whereas, the highfrequency information is typically clearly presented on highresolution.…”

“…To the best of our knowledge, only IMPLICTY (Stucker et al, 2022) proposed a training dataset for learning DSM which is from satellite images and not public open, and there is no benchmark of high-resolution UAV images for learning DSM in an implicit manner. To validate the efficacy of our deep-FG-DSM and make community that are interested in dealing with UAV images be able to access relevant benchmark, we generated an aerial image implicit reconstruction dataset 2 , named as And information of our UAVIIR and code can be found at https://github.com/3241674469/Deep-FG-DSM.…”

“…typically results in significant noise and fails to preserve the fine details that are often clearly presented on UAV images. Recently, learning-based implicit representation studies have gained increasing popularity, such as DeepSDF (Park et al, 2019) and IMPLICITY (Stucker et al, 2022), which, in principle, can yield infinite resolution DSM, and its practical resolution might be limited by factors such as input data resolution and the number of neurons in the network architecture. Thus, learningbased implicit representation could be a promising alternative for generating fine-gained DSM in complex and large-scale scenes.…”

“…CON refined object surface modeling by predicting the occupancy probability of input points, extending from single-object to implicit 3D reconstruction in large-scale indoor scenes. The latest work, IMPLICITY (Stucker et al, 2022), combined the network architectures of CON and PIFu for producing large area DSM from satellite imagery. Although IMPLICITY preserves local details of buildings, it * Corresponding Author focuses solely on the 3D point features, neglecting the local characteristics among neighboring points, which may lead to deformations in building shapes and rough edges for very sparse point cloud, deviating from their authentic forms.…”