Scale Sequence Joint Deep Learning (SS-JDL) for land use and land cover classification

Abstract

Choosing appropriate scales for remotely sensed image classification is extremely important yet still an open question in relation to deep convolutional neural networks (CNN), due to the impact of spatial scale (i.e., input patch size) on the recognition of ground objects. Currently, the optimal scale selection processes are extremely cumbersome and timeconsuming requiring repetitive experiments involving trial-and-error procedures, which significantly reduces the practical utility of the corresponding classification methods. This issue is crucial when trying to classify large-scale land use (LU) and land cover (LC) jointly (Zhang et al., 2019). In this paper, a simple and parsimonious scale sequence joint deep learning (SS-JDL) method is proposed for joint LU and LC classification, in which a sequence of scales is embedded in the iterative process of fitting the joint distribution implicit in the joint deep learning (JDL) method, thus, replacing the previous paradigm of scale selection. The sequence of scales, derived autonomously and used to define the CNN input patch sizes, provides consecutive information transmission from small-scale features to large-scale representations, and from simple LC states to complex LU characterisations. The effectiveness 2 of the novel SS-JDL method was tested on aerial digital photography of three complex and heterogeneous landscapes, two in Southern England (Bournemouth and Southampton) and one in North West England (Manchester). Benchmark comparisons were provided in the form of a range of LU and LC methods, including the state-of-the-art joint deep learning (JDL) method. The experimental results demonstrated that the SS-JDL consistently outperformed all of the state-of-the-art baselines in terms of both LU and LC classification accuracies, as well as computational efficiency. The proposed SS-JDL method, therefore, represents a fast and effective implementation of the state-of-the-art JDL method. By creating a single, unifying joint distribution framework for classifying higher order feature representations, including LU, the SS-JDL method has the potential to transform the classification paradigm in remote sensing, and in machine learning more generally.

Deep learning for processing and analysis of remote sensing big data: a technical review

“…Pixelwise LULC classification, assigning land-type labels to each pixel in an image, has a similar technical procedure as semantic segmentation in computer vision. Recent pixelwise LULC classification using semantic segmentation networks has achieved great advances (Tong et al, 2020;Zhang et al, 2020aZhang et al, , 2019a. In particular, to further explore spatial and graph topological information, GNNs have been employed in LULC classification tasks on hyperspectral images (Mou et al, 2020;Qin et al, 2018;Wan, Gong, & Zhong et al, 2019;Wan et al, 2020;Wang, Ma, Chen, & Du, 2021), very high-resolution satellite images (Cui et al, 2021;Khan, Chaudhuri, Banerjee, & Chaudhuri, 2019;Liu, Kampffmeyer, & Jenssen et al, 2020b;Ouyang & Li, 2021), and time-series images (Censi et al, 2021).…”

A Multi-Level Output-Based DBN Model for Fine Classification of Complex Geo-Environments Area Using Ziyuan-3 TMS Imagery

“…Furthermore, deep learning algorithms are increasingly popular in remote sensing classification because depth and discriminating features can be extracted layer-by-layer [ 37 , 38 ]. Zhang et al [ 39 ] proposed a scale sequence joint deep learning method by incorporating a sequence of scales in a single unified modeling framework for LULC classification. Chen et al [ 37 ] proposed a novel attention-driven context encoding network method for coastal land cover classification from high-resolution remote sensing images.…”

PCNet: Cloud Detection in FY-3D True-Color Imagery Using Multi-Scale Pyramid Contextual Information

“…Beyond machine learning, the deep-learning neural network has attracted much attention from academia since Alexnet achieved excellent performance on Imagenet in 2012 [20]. Owing to the powerful feature extraction capabilities and minimal manual intervention, the neural network has been widely used in target detection [21][22][23][24], land cover/use [25][26][27][28] and change detection tasks [29][30][31][32]. Based on automatically extracted geographical and semantic features from a network, the performance of the cloud-detection algorithm is accelerated by deep-learning techniques in image classification tasks [33].…”