Fusion Classification of HSI and MSI Using a Spatial-Spectral Vision Transformer for Wetland Biodiversity Estimation

Gao, Yunhao; Song, Xiukai; Li, Wei; Wang, Jianbu; He, Jianlong; Jiang, Xiangyang; Feng, Yinyin

doi:10.3390/rs14040850

Cited by 17 publications

(9 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-LU/LC Wetland biodiversity estimation ZiYhis1-02D-HSI (30 m) (2) No - [467,468] [193] LU/LC Wetland biodiversity estimation ZiYuan1-02D-MSI (10 m) No -[468] - (1) This parameter was mapped and monitored together with other parameters; (2) hyperspectral data. -LAI (1) Mangrove ecosystems MODIS Yes --[470] LAI (1) Mangrove ecosystems Landsat (15-30 m) No -[365] -LAI (1) Mangrove ecosystems Sentinel-2 (10-20-60 m) No - [365] [123] LAI (1) Mangrove ecosystems SPOT (~10-20 m) No -[365] -LAI (1) Mangrove ecosystems WorldView-2 (~0.5-2 m) No --[123] LAI (1) Mangrove ecosystems UAV No --[123] LAI (1) Seagrass Landsat (15-30 m) No - [233] - (1) Coastline change Landsat (15…”

Section: Parametermentioning

confidence: 99%

“…The eligible papers that mapped and/or monitored vegetation species. Coastal vegetation (1) Coastal vulnerability assessment Sentinel-2 (10-20-60 m) No -- [166] Coastal vegetation (1) Habitat GaoFen2 (0.8-3.2 m) No --[559] Coastal vegetation (1) Habitat Google Earth image No --[164] Coastal vegetation (1) Habitat Landsat (15-30 m) No --[424,559] Coastal vegetation (1) Habitat RapidEye (5 m) No -- [424] Coastal vegetation (1) Wetland Biodiversity Estimation ZiYuan1-02D-HIS (30 m) (2) No - [467] -Coastal vegetation (1) Wetland Biodiversity Estimation ZiYuan1-02D-MSI (10 m) No -[467] -Invasive species (1) Coastal vulnerability assessment Landsat (15-30 m) No [55] - [413] Invasive species (1) Invasive alien species Landsat (15-30 m) No --[520] Invasive species (1) Invasive alien species Sentinel-2 (10-20-60 m) No --[473,520] Invasive species (1) Invasive alien species UAV-MSI No [161] --Invasive species (1) Invasive alien species UAV No [161] --Invasive species (1) LU/LC change Google Earth image No --[437] Invasive species (1) LU/LC change Landsat (15-30 m) No --[437] Invasive species (1) Mangrove ecosystems ALOS-2 No -[365] -Invasive species (1) Mangrove ecosystems Landsat (15-30 m) No -[365] -Invasive species (1) Mangrove ecosystems SPOT (~10-20 m) No -[365] -Invasive species (1) Tidal flat Sentinel-2 (10-20-60 m) No -[453] -Riparian species (1) Invasive alien species Landsat (15-30 m) No -[432] -Salt marsh species (1) Habitat Sentinel-1 (~10 m) No --[560] Salt marsh species (1) Habitat Sentinel-2 (10-20-60 m) No --[428,560] Salt marsh species (1) Habitat UAV-Lidar No -[142] -Salt marsh species (1) Habitat UAV-MSI No -[142] -Wetland species (1) Habitat Sentinel-1 (~10 m) No …”

Section: Parametermentioning

confidence: 99%

See 1 more Smart Citation

Remote Data for Mapping and Monitoring Coastal Phenomena and Parameters: A Systematic Review

Cavalli

2024

Remote Sensing

View full text Add to dashboard Cite

Since 1971, remote sensing techniques have been used to map and monitor phenomena and parameters of the coastal zone. However, updated reviews have only considered one phenomenon, parameter, remote data source, platform, or geographic region. No review has offered an updated overview of coastal phenomena and parameters that can be accurately mapped and monitored with remote data. This systematic review was performed to achieve this purpose. A total of 15,141 papers published from January 2021 to June 2023 were identified. The 1475 most cited papers were screened, and 502 eligible papers were included. The Web of Science and Scopus databases were searched using all possible combinations between two groups of keywords: all geographical names in coastal areas and all remote data and platforms. The systematic review demonstrated that, to date, many coastal phenomena (103) and parameters (39) can be mapped and monitored using remote data (e.g., coastline and land use and land cover changes, climate change, and coastal urban sprawl). Moreover, the authors validated 91% of the retrieved parameters, retrieved from remote data 39 parameters that were mapped or monitored 1158 times (88% of the parameters were combined together with other parameters), monitored 75% of the parameters over time, and retrieved 69% of the parameters from several remote data and compared the results with each other and with available products. They obtained 48% of the parameters using different methods, and their results were compared with each other and with available products. They combined 17% of the parameters that were retrieved with GIS and model techniques. In conclusion, the authors addressed the requirements needed to more effectively analyze coastal phenomena and parameters employing integrated approaches: they retrieved the parameters from different remote data, merged different data and parameters, compared different methods, and combined different techniques.

show abstract

Section: Parametermentioning

confidence: 99%

Section: Parametermentioning

confidence: 99%

Remote Data for Mapping and Monitoring Coastal Phenomena and Parameters: A Systematic Review

Cavalli

2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The fused information is then fed into a two-branch CNN to perform classification. Gao et al [23] created a technique for identifying the land cover of complicated wetlands with patchy, mixed vegetation. First, CNN is utilized to merge the HSI and multispectral characteristics.…”

Section: Literature Surveymentioning

confidence: 99%

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

Mani,

Raguttapalli Chowdareddy

2024

IJECE

View full text Add to dashboard Cite

Because of its spectral-spatial and temporal resolution of greater areas, hyperspectral imaging (HSI) has found widespread application in the field of object classification. The HSI is typically used to accurately determine an object's physical characteristics as well as to locate related objects with appropriate spectral fingerprints. As a result, the HSI has been extensively applied to object identification in several fields, including surveillance, agricultural monitoring, environmental research, and precision agriculture. However, because of their enormous size, objects require a lot of time to classify; for this reason, both spectral and spatial feature fusion have been completed. The existing classification strategy leads to increased misclassification, and the feature fusion method is unable to preserve semantic object inherent features; This study addresses the research difficulties by introducing a hybrid spectral-spatial fusion (HSSF) technique to minimize feature size while maintaining object intrinsic qualities; Lastly, a soft-margins kernel is proposed for multi-layer deep support vector machine (MLDSVM) to reduce misclassification. The standard Indian pines dataset is used for the experiment, and the outcome demonstrates that the HSSF-MLDSVM model performs substantially better in terms of accuracy and Kappa coefficient.

show abstract

“…There have been few studies that have used ViT to process temporal remote sensing images. For example, Gao et al [47] designed a spatio-spectral vision transformer (SSViT) to extract sequential relationships from fused hyperspectral and multispectral images, and Chen et al [48] presented a transformer-based structure for multi-temporal remote sensing interpretation. However, there are currently no studies that have comprehensively evaluated the performance of different ViT structures, as well as high-dimensional CNNs and RNNs, all of which have the ability to process time series in land-cover classification from multi-temporal remote sensing images.…”

Section: Introductionmentioning

confidence: 99%

CNN, RNN, or ViT? An Evaluation of Different Deep Learning Architectures for Spatio-Temporal Representation of Sentinel Time Series

Zhao

2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Rich information in multi-temporal satellite images can facilitate pixel-level land cover classification. However, what is the most suitable deep learning architecture for high-dimension spatio-temporal representation of remote sensing time-series remains unclear. In this study, we theoretically analyzed the different mechanisms of the different deep learning structures, including the commonly used convolutional neural network (CNN), the high-dimension CNN (3D CNN), the recurrent neural network (RNN), and the newest vision transformer (ViT), with regard to learning and representing the temporal information for spatiotemporal data. The performance of the different models was comprehensively evaluated on large-scale Sentinel-1 and Sentinel-2 time-series images covering the whole of Slovenia. Several observations can be made. Firstly, the 3D CNN, long short-term memory (LSTM), and ViT, which all have specific structures that preserve temporal information, can effectively extract the spatiotemporal information, with the 3D CNN and ViT showing the best performance.Secondly, the performance of the 2D CNN, in which the temporal information is collapsed, is lower than that of the 3D CNN, LSTM, and ViT; however, it significantly outperforms the conventional methods of random forest (RF) and XGBoost. We also observed that using both optical and synthetic aperture radar (SAR) images performs almost the same as using only optical images, indicating that the information that can be extracted from optical images is sufficient for land-cover classification. However, when optical imaging is affected by poor weather, SAR images, as a beneficial supplement, can provide satisfactorily classification results. Finally, the modern deep learning methods can effectively overcome the disadvantages in imaging conditions where parts of an image or images of some periods are missing. The testing data are available at gpcv.whu.edu.cn/data.

show abstract

Fusion Classification of HSI and MSI Using a Spatial-Spectral Vision Transformer for Wetland Biodiversity Estimation

Cited by 17 publications

References 43 publications

Remote Data for Mapping and Monitoring Coastal Phenomena and Parameters: A Systematic Review

Remote Data for Mapping and Monitoring Coastal Phenomena and Parameters: A Systematic Review

Hyperspectral object classification using hybrid spectral-spatial fusion and noise tolerant soft-margin technique

CNN, RNN, or ViT? An Evaluation of Different Deep Learning Architectures for Spatio-Temporal Representation of Sentinel Time Series

Contact Info

Product

Resources

About