Dense Semantic Labeling of Very-High-Resolution Aerial Imagery and LiDAR with Fully-Convolutional Neural Networks and Higher-Order CRFs

Liu, Yansong; Piramanayagam, Sankaranarayanan; Monteiro, Sildomar T.; Saber, Eli

doi:10.1109/cvprw.2017.200

Cited by 84 publications

(78 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best result, in terms of overall accuracy, was 90.3% achieved by DLR 9 [22] and GSN3 [27]. Our best result (UFMG 4) appears in fifth place by yielding 89.4% of overall accuracy, outperforming several methods, such as ADL 3 [49] and RIT L8 [50], that also tried to aggregate multi-context information. However, as can be seen in Table XI and Figure 10a, while the other approaches have a larger number of trainable parameters, our network has only 2 millions, which makes it less pruned to overfitting and, consequently, easier to train, showing that the proposed method really helps in extracting all feasible information of the data even if using limited architectures (in terms of parameters).…”

Section: F Performance Analysismentioning

confidence: 79%

See 1 more Smart Citation

Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks

Nogueira

Mura

Chanussot

et al. 2019

IEEE Trans. Geosci. Remote Sensing

127

View full text Add to dashboard Cite

Semantic segmentation requires methods capable of learning high-level features while dealing with large volume of data. Towards such goal, Convolutional Networks can learn specific and adaptable features based on the data. However, these networks are not capable of processing a whole remote sensing image, given its huge size. To overcome such limitation, the image is processed using fixed size patches. The definition of the input patch size is usually performed empirically (evaluating several sizes) or imposed (by network constraint). Both strategies suffer from drawbacks and could not lead to the best patch size. To alleviate this problem, several works exploited multi-context information by combining networks or layers. This process increases the number of parameters resulting in a more difficult model to train. In this work, we propose a novel technique to perform semantic segmentation of remote sensing images that exploits a multi-context paradigm without increasing the number of parameters while defining, in training time, the best patch size. The main idea is to train a dilated network with distinct patch sizes, allowing it to capture multi-context characteristics from heterogeneous contexts. While processing these varying patches, the network provides a score for each patch size, helping in the definition of the best size for the current scenario. A systematic evaluation of the proposed algorithm is conducted using four high-resolution remote sensing datasets with very distinct properties. Our results show that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to state-of-the-art methods.

show abstract

Section: F Performance Analysismentioning

confidence: 79%

“…The proposed work achieved competitive results, appearing in third place according to the overall accuracy. DST 5 [20] and RIT L7 [50] are the best result in terms of overall accuracy. However, they have a larger number of trainable parameters when compared to our proposed networks, as seen in Figure 10b.…”

Section: F Performance Analysismentioning

confidence: 99%

Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks

Nogueira

Mura

Chanussot

et al. 2019

IEEE Trans. Geosci. Remote Sensing

127

View full text Add to dashboard Cite

show abstract

“…Saito and Aoki [46] used CNN based approaches for building and road extraction. Liu et al [34] used FCN-8 segmentation network analyzing IR, R and G data with 5 convolutional layers and augmentation with a model based on nDSM (normalized Digital Surface Model) and NDVI. Inria competition solutions described in [29] used U-Net or SegNet approaches to segmentation.…”

Section: Building Detectionmentioning

confidence: 99%

DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images

Demir

Koperski

Lindenbaum

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

786

428

View full text Add to dashboard Cite

Figure 1: DeepGlobe Challenges: Example road extraction, building detection, and land cover classification training images superimposed on corresponding satellite images. AbstractWe present the DeepGlobe 2018 Satellite Image Understanding Challenge, which includes three public competitions for segmentation, detection, and classification tasks on satellite images ( Figure 1). Similar to other challenges in computer vision domain such as DAVIS[21] and COCO[33], DeepGlobe proposes three datasets and corresponding evaluation methodologies, coherently bundled in three competitions with a dedicated workshop co-located with CVPR 2018.We observed that satellite imagery is a rich and structured source of information, yet it is less investigated than everyday images by computer vision researchers. However, bridging modern computer vision with remote sensing data analysis could have critical impact to the way we understand our environment and lead to major breakthroughs in global urban planning or climate change research. Keeping such bridging objective in mind, DeepGlobe aims to bring together researchers from different domains to raise awareness of remote sensing in the computer vision community and vice-versa. We aim to improve and evaluate state-of-the-art satellite image understanding approaches, which can hopefully serve as reference benchmarks for future research in the same topic. In this paper, we analyze characteristics of each dataset, define the evaluation criteria of the competitions, and provide baselines for each task.

show abstract

“…Data fusion is a fields where UGMs could have a major impact. They have been successfully used to fuse multi-modal images for land cover classification [48,84]. However, more interesting applications would arise from the fusion of ground level data, such as digital maps (e.g., OpenStreetMaps) and geotagged data (e.g., photos, online reviews), with the remotely sensed images.…”

Section: Discussionmentioning

confidence: 99%

A tutorial on modelling and inference in undirected graphical models for hyperspectral image analysis

Gewali

Monteiro

2018

International Journal of Remote Sensing

Self Cite

View full text Add to dashboard Cite

Undirected graphical models have been successfully used to jointly model the spatial and the spectral dependencies in earth observing hyperspectral images. They produce less noisy, smooth, and spatially coherent land cover maps and give top accuracies on many datasets. Moreover, they can easily be combined with other state-of-the-art approaches, such as deep learning. This has made them an essential tool for remote sensing researchers and practitioners. However, graphical models have not been easily accessible to the larger remote sensing community as they are not discussed in standard remote sensing textbooks and not included in the popular remote sensing software and toolboxes. In this tutorial, we provide a theoretical introduction to Markov random fields and conditional random fields based spatial-spectral classification for land cover mapping along with a detailed step-by-step practical guide on applying these methods using freely available software. Furthermore, the discussed methods are benchmarked on four public hyperspectral datasets for a fair comparison among themselves and easy comparison with the vast number of methods in literature which use the same datasets. The source code necessary to reproduce all the results in the paper is published on-line to make it easier for the readers to apply these techniques to different remote sensing problems.illumination, shadows, purity of pixels, viewing geometry, atmospheric conditions, and noise across the image. This problem can be alleviated by combining spatial contextual information with the spectral information. Spatial-spectral classificationSpatial information can be utilized together with spectral information to produce more accurate and spatially coherent land cover maps [24]. Land covers in the environment tend to be much larger than the ground pixel size of the sensors leading to regions of pixels belonging to a common material class. Additionally, some land cover classes are more likely to exists in close vicinity than others and some land cover classes are highly unlikely to occur together. This leads to strong relationships between the neighboring pixel labels in an image. For example, if a pixel belongs to a class, say building, there is a high probability that the surrounding pixels also belong to the same class, building. Similarly, the probability of neighboring pixels of a building pixel belonging to the road class or the parking lot class is typically much higher than them belonging to the forest class or the bare soil class in an urban scene. These kind of relationships can be exploited by spatial-spectral classifiers. Even though there are exceptions, e.g., [12], [82], and [45], the vast majority of spatial-spectral classifiers can be categorized into two distinct groups-methods that perform spatial-spectral feature extraction followed by pixel-wise classification and methods that combine undirected graphical model and pixel-wise classification.Spatial-spectral feature extraction utilizes the spectra of the neighborhood of pixels around th...

show abstract

Dense Semantic Labeling of Very-High-Resolution Aerial Imagery and LiDAR with Fully-Convolutional Neural Networks and Higher-Order CRFs

Cited by 84 publications

References 29 publications

Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks

Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks

DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images

A tutorial on modelling and inference in undirected graphical models for hyperspectral image analysis

Contact Info

Product

Resources

About