Multisensor Earth Observation Image Classification Based on a Multimodal Latent Dirichlet Allocation Model

Abstract

Many previous researches have already shown the advantages of multi-sensor land-cover classification. Here, we propose an innovative land-cover classification approach based on learning a joint latent model of Synthetic Aperture Radar (SAR) and multispectral satellite images using multimodal Latent Dirichlet Allocation (mmLDA), a probabilistic generative model. It has already been successfully applied to various other problems dealing with multimodal data. For our experiments, we chose overlapping SAR and multispectral images of two regions of interest. The images were tiled into patches and their local primitive features were extracted. Then each image patch is represented by SAR and multispectral Bag-of-Words (BoW) models. The BoW values are both fed to the mmLDA, resulting in a joint latent data model. A qualitative and quantitative validation of the topics based on ground truth data demonstrate that the land-cover categories of the regions are correctly classified, outperforming the topics obtained by using individual single modality data.

Multimodal Probabilistic Latent Semantic Analysis for Sentinel-1 and Sentinel-2 Image Fusion

“…2) Berlin [10] contains two Sentinel-1B and Sentinel-2A data products from Berlin (Germany), captured on May 26 and 27, 2017, respectively, which cover the area between the (52.78°N, 12.45°E) upper left coordinates and (52.26°N, 13.67°E) lower right coordinates.…”

“…Indeed, it was not until recently that a multi-modal topic model was successfully applied to fuse SAR and MSI data. Specifically, Bahmanyar et al presented in [10] a multi-sensor land-cover classification technique using a visual bag-of-words (vBoW) characterization scheme together with a multi-modal variant of the Latent Dirichlet Allocation (LDA) model [11] which makes use of two different vocabularies to jointly represent SAR and MSI data modalities. Despite the potential of this recent LDA-based fusion approach to outperform individual single modality data, LDA is not the only type of topic model available in the literature and analyzing the effect of using different kinds of probabilistic topic models for fusing SAR and MSI remotely sensed data still remains an open-ended issue.…”

Combining Deep and Shallow Neural Networks with Ad Hoc Detectors for the Classification of Complex Multi-Modal Urban Scenes

“…The topic vectors are multi-modal high-level features computed by applying multi-modal latent Dirichlet allocation (mmLDA) [1] to the bag-of-words (BoW) model of the HS (1 m GSD) and RGB (50 cm GSD) images. The mmLDA discovers the joint model latency as a set of so-called topics and represents each image as topic mixtures.…”

“…Therefore, it is difficult to find a single classifier which correctly identifies all the different classes in such tasks. 1 The authors would like to thank the National Center for Airborne Laser Mapping and the Hyperspectral Image Analysis Laboratory at the University of Houston for acquiring and providing the data used in this study, and the IEEE GRSS Image Analysis and Data Fusion Technical Committee. Data available at http://www.grss-ieee.org/community/technical-committees/datafusion/data-fusion-contest Recently, classifiers based on deep learning have proven very promising in capturing the relevant features from a wide variety of classes; however, these may over-rely on higher order interactions among the pixels composing an object.…”

Joint SAR Image Time Series and PSInSAR Data Analytics: An LDA Based Approach

“…One of the algorithms that proved to be very efficient for multispectral SITS analysis [17], but also for single scene classification, regardless the type of imagery [18][19][20], is the Latent Dirichlet Allocation (LDA) model. The secret lies in the way one defines analogies between the analyzed data and notions like word, document, and corpus.…”