Sat-Mesh: Learning Neural Implicit Surfaces for Multi-View Satellite Reconstruction

Abstract

Automatic reconstruction of surfaces from satellite imagery is a hot topic in computer vision and photogrammetry. State-of-the-art reconstruction methods typically produce 2.5D elevation data. In contrast, we propose a one-stage method directly generating a 3D mesh model from multi-view satellite imagery. We introduce a novel Sat-Mesh approach for satellite implicit surface reconstruction: We represent the scene as a continuous signed distance function (SDF) and leverage a volume rendering framework to learn the SDF values. To address the challenges posed by lighting variations and inconsistent appearances in satellite imagery, we incorporate a latent vector in the network architecture to encode image appearances. Furthermore, we introduce a multi-view stereo constraint to enhance surface quality. This constraint minimizes the similarity between image patches to optimize the position and orientation of the SDF surface. Experimental results demonstrate that our method achieves superior visual quality and quantitative accuracy in generating mesh models. Moreover, our approach can learn seasonal variations in satellite imagery, resulting in texture mesh models with different and consistent seasonal appearances.

SatelliteRF: Accelerating 3D Reconstruction in Multi-View Satellite Images with Efficient Neural Radiance Fields

“…Refs. [11,16,[37][38][39] have explored satellite 3D reconstruction through the neural radiance field method. Derksen et al [11] took the lead in exploring remote sensing 3D reconstruction based on neural radiance fields and proposed S-NeRF, which is a significant advancement in the discipline of remote sensing 3D reconstruction based on neural radiance fields.…”

“…In addition, Sat-NeRF [16] introduces the RPC camera model to more accurately calculate the origin and direction of rays for ray marching in the NeRF, and it solves the problem of transient phenomena that are not easily explained by the sun's position through the shadow-aware irradiance model and uncertainty weights. Sat-Mesh [39], on the other hand, uses the MLP to learn the sign distance function (SDF) and integrates it within the volume-rendering framework to achieve multi-view satellite reconstruction. Season-NeRF [37] enables the NeRF model to learn and render seasonal features by including time as an additional input variable.…”

Camera and LiDAR Fusion for Urban Scene Reconstruction and Novel View Synthesis via Voxel-Based Neural Radiance Fields

“…3D reconstruction [5,6] can be used not only for data augmentation but also for direct 3D modeling of urban scenes [7]. Specifically, in the remote sensing mapping [8][9][10][11][12], it can generate high-precision digital surface models using multi-view satellite images [13,14] and combine the diversity of virtual environments with the richness of the real-world, generating more controllable and realistic data than simulation data.…”