Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks

Abstract

Zero-shot learning (ZSL) is an approach to classify objects unseen during the training phase and shown to be useful for real-world applications, especially when there is a lack of sufficient training data. Only a limited amount of works has been carried out on ZSL, especially in the field of remote sensing. This research investigates the use of a convolutional neural network (CNN) as a feature extraction and classification method for land cover mapping using high-resolution orthophotos. In the feature extraction phase, we used a CNN model with a single convolutional layer to extract discriminative features. In the second phase, we used class attributes learned from the Word2Vec model (pre-trained by Google News) to train a second CNN model that performed class signature prediction by using both the features extracted by the first CNN and class attributes during training and only the features during prediction. We trained and tested our models on datasets collected over two subareas in the Cameron Highlands (training dataset, first test dataset) and Ipoh (second test dataset) in Malaysia. Several experiments have been conducted on the feature extraction and classification models regarding the main parameters, such as the network’s layers and depth, number of filters, and the impact of Gaussian noise. As a result, the best models were selected using various accuracy metrics such as top-k categorical accuracy for k = [1,2,3], Recall, Precision, and F1-score. The best model for feature extraction achieved 0.953 F1-score, 0.941 precision, 0.882 recall for the training dataset and 0.904 F1-score, 0.869 precision, 0.949 recall for the first test dataset, and 0.898 F1-score, 0.870 precision, 0.838 recall for the second test dataset. The best model for classification achieved an average of 0.778 top-one, 0.890 top-two and 0.942 top-three accuracy, 0.798 F1-score, 0.766 recall and 0.838 precision for the first test dataset and 0.737 top-one, 0.906 top-two, 0.924 top-three, 0.729 F1-score, 0.676 recall and 0.790 precision for the second test dataset. The results demonstrated that the proposed ZSL is a promising tool for land cover mapping based on high-resolution photos.

Generalized Zero-Shot Domain Adaptation for Unsupervised Cross-Domain PolSAR Image Classification

“…That is, it is infeasible to identify the new classes of target domain by samples from source domain. To realize uGZSDA, not only the CDS problem between the source and target domain should be overcome, but also the unseen classes should be inferred according to the polarimetric distribution characteristics and semantic spaces [24], [30].…”

“…At present, the semantic information used in remote sensing image ZSL and GZSL mainly comes from Word2Vec or SUN attributes [30], [39]. However, these semantics from natural languages and natural image models are not befitting for PolSAR description.…”

“…Recently, zero-shot domain adaptation (ZSDA) and generalized zero-shot domain adaptation (GZSDA) have been developed and concerned [21]- [23] in the computer vision field, as promising solutions to the above problems. They have demonstrated the ability to infer and identify unseen categories in the target domain by using the seen samples and semantic information [24]- [28]. However, both ZSDA and GZSDA need labeled seen samples from source and target domain.…”

Unknown SAR Target Identification Method Based on Feature Extraction Network and KLD–RPA Joint Discrimination

“…Traditional end to end SAR image unknown target recognition methods [3,6,26,29,31] bear problems such as target feature closure, lack of effective processing, and operation for target features. When faced with unknown targets that are not involved in training, these traditional methods cannot effectively identify the unknown targets.…”

“…For those unknown SAR targets that are not involved in training, the above methods are completely unable to accurately identify. Zero-shot learning (ZSL) is a technique for learning and discriminating unknown sample data [21][22][23][24][25][26]. Compared with other DL based methods, the main characteristic of ZSL is that there is no intersection between the training sample set and the test sample set.…”