A tutorial on modelling and inference in undirected graphical models for hyperspectral image analysis

Abstract

Undirected graphical models have been successfully used to jointly model the spatial and the spectral dependencies in earth observing hyperspectral images. They produce less noisy, smooth, and spatially coherent land cover maps and give top accuracies on many datasets. Moreover, they can easily be combined with other state-of-the-art approaches, such as deep learning. This has made them an essential tool for remote sensing researchers and practitioners. However, graphical models have not been easily accessible to the larger remote sensing community as they are not discussed in standard remote sensing textbooks and not included in the popular remote sensing software and toolboxes. In this tutorial, we provide a theoretical introduction to Markov random fields and conditional random fields based spatial-spectral classification for land cover mapping along with a detailed step-by-step practical guide on applying these methods using freely available software. Furthermore, the discussed methods are benchmarked on four public hyperspectral datasets for a fair comparison among themselves and easy comparison with the vast number of methods in literature which use the same datasets. The source code necessary to reproduce all the results in the paper is published on-line to make it easier for the readers to apply these techniques to different remote sensing problems.illumination, shadows, purity of pixels, viewing geometry, atmospheric conditions, and noise across the image. This problem can be alleviated by combining spatial contextual information with the spectral information. Spatial-spectral classificationSpatial information can be utilized together with spectral information to produce more accurate and spatially coherent land cover maps [24]. Land covers in the environment tend to be much larger than the ground pixel size of the sensors leading to regions of pixels belonging to a common material class. Additionally, some land cover classes are more likely to exists in close vicinity than others and some land cover classes are highly unlikely to occur together. This leads to strong relationships between the neighboring pixel labels in an image. For example, if a pixel belongs to a class, say building, there is a high probability that the surrounding pixels also belong to the same class, building. Similarly, the probability of neighboring pixels of a building pixel belonging to the road class or the parking lot class is typically much higher than them belonging to the forest class or the bare soil class in an urban scene. These kind of relationships can be exploited by spatial-spectral classifiers. Even though there are exceptions, e.g., [12], [82], and [45], the vast majority of spatial-spectral classifiers can be categorized into two distinct groups-methods that perform spatial-spectral feature extraction followed by pixel-wise classification and methods that combine undirected graphical model and pixel-wise classification.Spatial-spectral feature extraction utilizes the spectra of the neighborhood of pixels around th...

A Bilevel Contextual MRF Model for Supervised Classification of High Spatial Resolution Remote Sensing Images

“…1) pixel level classifier: SVMCK classifier [6]; 2) filtering-based classifiers: LBP with decision fusion classifier (LBP-DF) [14], multiple-feature-based adaptive spare representation (MFASR) [55]; 3) object-based classifiers: SVMMRF with object-oriented voting (SVM-OO) [39]; 4) MRF-based classifiers: SVM classifier with MRF (SVMMRF) [34], superpixel level MRF classifier (SuperMRF) [29], sparse MLR with weighted MRF (SMLR-WMRF) [31], detail preserving smoothing classifier based on CRF (DPSCRF) [41]; 5) deep learning methods: CNN [17] and SSRN [18]. The CNN model contains three convolutional layers with a 5 × 5 × 128 convolutional kernels.…”

“…Specifically, both MRF and CRF based methods are able to incorporate labeled data and observed spatial contextual feature into an integrated framework. The CRF is a type of MRF whose clique potential is conditioned on input features [29], [30].…”

Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery

“…The expressiveness of the UGMs is controlled by the structure of the graph and energy functions defined over the graph's cliques. The most common type of UGM used for HSI classification is a pairwise model [11]. It defines the joint distribution of the pixel labels of an image as…”

“…EarthMapper includes grid-structured and fully-connected pairwise UGMs for post-processing. The grid-structured model is the most common type used in HSI semantic segmentation, and a detailed description can be found in [11]. The fullyconnected model, as described in this letter, has not been previously explored for HSI segmentation.…”

“…UGMs have been widely used in HSI classification to produce smooth, coherent segmentation maps by enforcing spatial contexts [11]. Their main advantage is that they can be easily combined with other classification approaches as a post-processing method that improves accuracy.…”