SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

Li, Xin; Wang, Taoyang; Hao, Cui; Zhang, Guo; Cheng, Qian; Dong, Tiancheng; Jiang, Binbin

doi:10.1109/jstars.2022.3196383

Cited by 9 publications

(4 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Registering two markedly dissimilar images poses a formidable challenge, as elucidated in our examination of the failure instances in Section 4.4.6. When the disparities between the two images slated for registration lie beyond the spectrum of variations encompassed by the training data, it may be necessary to resort to initial registration techniques such as geometric registration [5,50,51], block-based registration [52], or the integration of prior information, such as Digital Elevation Models (DEMs) and Ground Control Points (GCPs). In scenarios where neural-network-based approaches persist in being utilized, it becomes imperative to expand the scope of variations encompassed by the training dataset.…”

Section: Discussionmentioning

confidence: 99%

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

Zhou,

Wang,

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

For Synthetic Aperture Radar (SAR) image registration, successive processes following feature extraction are required by both the traditional feature-based method and the deep learning method. Among these processes, the feature matching process—whose time and space complexity are related to the number of feature points extracted from sensed and reference images, as well as the dimension of feature descriptors—proves to be particularly time consuming. Additionally, the successive processes introduce data sharing and memory occupancy issues, requiring an elaborate design to prevent memory leaks. To address these challenges, this paper introduces the OptionEM-based reinforcement learning framework to achieve end-to-end SAR image registration. This framework outputs registered images directly without requiring feature matching and the calculation of the transformation matrix, leading to significant processing time savings. The Transformer architecture is employed to learn image features, while a correlation network is introduced to learn the correlation and transformation matrix between image pairs. Reinforcement learning, as a decision process, can dynamically correct errors, making it more-efficient and -robust compared to supervised learning mechanisms such as deep learning. We present a hierarchical reinforcement learning framework combined with Episodic Memory to mitigate the inherent problem of invalid exploration in generalized reinforcement learning algorithms. This approach effectively combines coarse and fine registration, further enhancing training efficiency. Experiments conducted on three sets of SAR images, acquired by TerraSAR-X and Sentinel-1A, demonstrated that the proposed method’s average runtime is sub-second, achieving subpixel registration accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

Zhou,

Wang,

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In this section, Six representative matching algorithms (SIFT [19], SAR -SIFT [33], SAR PointNet [51], CFOG [22], RIFT [52], and MoTIF [53] ) including artificial feature matching algorithm and deep learning algorithm, were integrated with the framework first. Then different MAs were performed TPs matching from different RSI in Test Data 3.…”

Section: Framework Generality Verificationmentioning

confidence: 99%

MSRSI-TPMF: A Tie Points Matching Framework of Multisource Remote Sensing Images

Cheng,

Li,

Wang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

Remote sensing sensor platforms are typically located at a significant distance from the ground, ranging from several hundred meters to hundreds of kilometers. This means that, compared to natural images, remote sensing images (RSI) have larger coverage areas and more complex information. The larger size and data volume of RSI presents challenges for computer vision matching algorithms (MAs), making it difficult to apply them directly to RSI matching. Moreover, a matching framework for multi-source remote sensing images capable of large-scale processing by integrating multiple MAs with the entire RSI as input is presently lacking. This study proposes a Tie Points (TPs) Matching Framework of Multi-Source Remote Sensing Images (MSRSI-TPMF) based on the geometric and radiation characteristics of RSI. First, RSI is divided into different grids and undergo local geometry correction. Next, matching between slice images are performed by MAs. Finally, TPs are generated by mapping matched points in multiple slice images to the whole RSI using a geometric processing model. Six representative MAs including artificial feature matching algorithms and deep learning algorithms are integrated into the framework to match TPs from different RSI. Results demonstrate the extraction of TPs for multi-source RSI, validating the framework's efficacy. In addition, a large-scale TPs matching test for deep learning matching algorithm is performed by using 13 synthetic aperture radar (SAR) images (10m resolution) with TPs root mean square error (RMSE) of 0.368 pixels, further confirming the framework's reliability. Our framework is available at https://github.com/TobyChengV1/MSRSI-TPMF.

show abstract

“…Gain and offset values needed to be added to the image before calibration. Landsat8 OLI and GF-1 images were set according to the relevant parameters of the downloaded image file; this research uses the RPC module [33,34] in orthorectification, the RPC parameters included with GF-1 WFV data are used for correction, the DEM data required for correction is from ASTER satellite's 30 m spatial resolution GDEM product (https://www.gscloud.cn/ search?kw=GF-1 (accessed on 1 May 2022)), eliminating the geometric distortion caused by the influence of the mountain, and the deformation caused by the camera orientation; the atmospheric correction was performed on Landsat8 OLI and GF-1 WFV through the FLASSH atmospheric correction module [35,36]. As a result, the influence of external factors such as atmosphere and light on the image was eliminated, and a more accurate surface reflectance was obtained.…”

Section: Remote Sensing Image Dataset Preprocessingmentioning

confidence: 99%

A High Spatiotemporal Enhancement Method of Forest Vegetation Leaf Area Index Based on Landsat8 OLI and GF-1 WFV Data

et al. 2023

View full text Add to dashboard Cite

The leaf area index (LAI) is a crucial parameter for analyzing terrestrial ecosystem carbon cycles and global climate change. Obtaining high spatiotemporal resolution forest stand vegetation LAI products over large areas is essential for an accurate understanding of forest ecosystems. This study takes the northwestern part of the Inner Mongolia Autonomous Region (the northern section of the Greater Khingan Mountains) in northern China as the research area. It also generates the LAI time series product of the 8-day and 30 m forest stand vegetation growth period from 2013 to 2017 (from the 121st to the 305th day of each year). The Simulated Annealing-Back Propagation Neural Network (SA-BPNN) model was used to estimate LAI from Landsat8 OLI, and the multi-period GaoFen-1 WideField-View satellite images (GF-1 WFV) and the spatiotemporal adaptive reflectance fusion mode (STARFM) was used to predict high spatiotemporal resolution LAI by combining inversion LAI and Global LAnd Surface Satellite-derived vegetation LAI (GLASS LAI) products. The results showed the following: (1) The SA-BPNN estimation model has relatively high accuracy, with R2 = 0.75 and RMSE = 0.38 for the 2013 LAI estimation model, and R2 = 0.74 and RMSE = 0.17 for the 2016 LAI estimation model. (2) The fused 30 m LAI product has a good correlation with the LAI verification of the measured sample site (R2 = 0.8775) and a high similarity with the GLASS LAI product. (3) The fused 30 m LAI product has a high similarity with the GLASS LAI product, and compared with the GLASS LAI interannual trend line, it accords with the growth trend of plants in the seasons. This study provides a theoretical and technical reference for forest stand vegetation growth period LAI spatiotemporal fusion research based on high-score data, and has an important role in exploring vegetation primary productivity and carbon cycle changes in the future.

show abstract

SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

Cited by 9 publications

References 43 publications

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

MSRSI-TPMF: A Tie Points Matching Framework of Multisource Remote Sensing Images

A High Spatiotemporal Enhancement Method of Forest Vegetation Leaf Area Index Based on Landsat8 OLI and GF-1 WFV Data

Contact Info

Product

Resources

About