Abstract:The Rational Polynomial Camera (RPC) model can be used to describe a variety of image acquisition systems in remote sensing, notably optical and Synthetic Aperture Radar (SAR) sensors. RPC functions relate 3D to 2D coordinates and vice versa, regardless of physical sensor specificities, which has made them an essential tool to harness satellite images in a generic way. This article describes a terrain-independent algorithm to accurately derive a RPC model from a set of 3D-2D point correspondences based on a re… Show more
“…Note that the embedding vector t j is learned from the image index j during training. 1 We find that it is better to start using β after the second epoch, when the shadow-aware shading s is already well initialized. Otherwise the model may use β to overlook shadow areas instead of trying to explain them with s. Thus, we replace (8) with (5) in the first two epochs.…”
Section: Uncertainty Weighting For Transient Objectsmentioning
confidence: 85%
“…Sat-NeRF casts rays directly using the RPC camera models of a set of satellite images. The RPC model is widely used for optical satellite imagery, as it allows to describe complex acquisition systems independently of satellitespecific physical modeling [1,20]. Each RPC is defined by a projection function (to project 3D points onto image pixels) and its inverse, the localization function.…”
Section: Point Sampling From Satellite Rpc Modelsmentioning
We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogrammetry in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models, represented by rational polynomial coefficient (RPC) functions. The proposed method renders new views and infers surface models of similar quality to those obtained with traditional state-of-the-art stereo pipelines. Multi-date images exhibit significant changes in appearance, mainly due to varying shadows and transient objects (cars, vegetation). Robustness to these challenges is achieved by a shadow-aware irradiance model and uncertainty weighting to deal with transient phenomena that cannot be explained by the position of the sun. We evaluate Sat-NeRF using WorldView-3 images from different locations and stress the advantages of applying a bundle adjustment to the satellite camera models prior to training. This boosts the network performance and can optionally be used to extract additional cues for depth supervision.
“…Note that the embedding vector t j is learned from the image index j during training. 1 We find that it is better to start using β after the second epoch, when the shadow-aware shading s is already well initialized. Otherwise the model may use β to overlook shadow areas instead of trying to explain them with s. Thus, we replace (8) with (5) in the first two epochs.…”
Section: Uncertainty Weighting For Transient Objectsmentioning
confidence: 85%
“…Sat-NeRF casts rays directly using the RPC camera models of a set of satellite images. The RPC model is widely used for optical satellite imagery, as it allows to describe complex acquisition systems independently of satellitespecific physical modeling [1,20]. Each RPC is defined by a projection function (to project 3D points onto image pixels) and its inverse, the localization function.…”
Section: Point Sampling From Satellite Rpc Modelsmentioning
We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogrammetry in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models, represented by rational polynomial coefficient (RPC) functions. The proposed method renders new views and infers surface models of similar quality to those obtained with traditional state-of-the-art stereo pipelines. Multi-date images exhibit significant changes in appearance, mainly due to varying shadows and transient objects (cars, vegetation). Robustness to these challenges is achieved by a shadow-aware irradiance model and uncertainty weighting to deal with transient phenomena that cannot be explained by the position of the sun. We evaluate Sat-NeRF using WorldView-3 images from different locations and stress the advantages of applying a bundle adjustment to the satellite camera models prior to training. This boosts the network performance and can optionally be used to extract additional cues for depth supervision.
“…This model gives corresponding 3D/2D coordinates between the volume of interest (AOI plus height range) and the image. The correspondences are then used to adjust an RPC camera model using the RPCFIT tool [2], which fits an RPC model to the 3D/2D correspondences through a regularized least squares minimization.…”
Multi-view stereo reconstruction of scenes from satellite images is traditionally performed with a pair-wise stereovision approach: (1) multiple views are grouped into pairs, (2) each pair is processed by two-view stereo methods producing an elevation model or point cloud, lastly (3) the pairwise reconstructions are integrated and filtered to obtain a final result. These steps are organized in a pipeline and the end-to-end performance of reconstructions depends on the behavior of these steps. This work introduces two changes that increase the performance of the reconstructions: a new pair selection approach and a new integration method are presented. The new pair selection replaces commonly used heuristics with a principled criterion that predicts the completeness of a pair based on offline simulations. The presented integration method is based on an iterated bilateral filter. Experiments show that these changes yield a systematic improvement on the performance of the pipeline.
“…G2D ↔ G3D as input for (Akiki et al, 2021) to fit RPC + S 1 are registered and the RPC models adjusted, both choices will yield essentially the same 3D points for a range of altitudes centered around the surface. However, for points far from the surface we should start to observe a parallax due to the fact that the scenes are acquired from different positions along the orbit.…”
“…The previous procedure results into a set of 2D-to-3D point correspondences between S + 1 and the object space, thus the RPC fitting algorithm from (Akiki et al, 2021) can be applied to produce the final RPC + S 1 model.…”
Abstract. We propose a novel method to generate a single image product from a multi-image strip acquired by a push-frame satellite imaging system. The images of the push-frame strips are combined into a large scale mosaic simulating a perfect sensor geometry. The local camera models of the input images are leveraged to produce a new localization model that covers the output mosaic entirely. Among other applications, this simplifies the task of stereo reconstruction enormously: instead of treating multiple stereo pairs of small images, it is possible to reconstruct the entire area covered by the push-frame acquisition using a single pair of mosaics incorporating all the images. We test our method using strips of SkySat L1B scenes and denote the output images as L1B+. To evaluate the quality of the L1B+ images and their localization models, the stereo reconstructions obtained with L1B+ are compared with those obtained with L1B and with a lidar reference model.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.