A Vision Transformer Model for Convolution-Free Multilabel Classification of Satellite Imagery in Deforestation Monitoring

Kaselimi, Maria; Voulodimos, Athanasios; Daskalopoulos, Ioannis; Doulamis, Nikolaos; Doulamis, Anastasios

doi:10.1109/tnnls.2022.3144791

Cited by 55 publications

(26 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve their annotation, Deep Neural Networks (DNNs) with emphasis on Convolutional Neural Networks (CNNs) for image classification are considered [4]. They are compelling for object detection in remote sensing data, covering several applications including building extraction [5], deforestation [6], land cover change [7] and others.…”

Section: Introductionmentioning

confidence: 99%

Interpretable Deep Learning Framework for Land Use and Land Cover Classification in Remote Sensing Using SHAP

Τέμενος

Temenos

Kaselimi

et al. 2023

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

An interpretable deep learning framework for land use and land cover classification (LULC) in remote sensing using SHAP is introduced. It utilizes a compact CNN model for the classification of satellite images and then feeds the results to a SHAP deep explainer so as to strengthen the classification results. The proposed framework is applied to Sentinel-2 satellite images containing 27000 images of pixel size 64 × 64 and operates on three-band combinations, reducing the model's input data by 77% considering that 13 channels are available, while at the same time investigating on how different spectrum bands affect predictions on the dataset's classes. Experimental results on the EuroSAT dataset demonstrate the CNN's accurate classification with an overall accuracy of 94.72%, whereas the classification accuracy on three-band combinations on each of the dataset's classes highlights its improvement when compared to standard approaches with larger number of trainable parameters. The SHAP explainable results of the proposed framework shield the network's predictions by showing correlation values that are relevant to the predicted class, thereby improving the classifications occurring in urban and rural areas with different land uses in the same scene.

show abstract

Section: Introductionmentioning

confidence: 99%

Interpretable Deep Learning Framework for Land Use and Land Cover Classification in Remote Sensing Using SHAP

Τέμενος

Temenos

Kaselimi

et al. 2023

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

show abstract

“…Transformers have been used for multiple computer vision tasks, namely scene classification [19], [20], change detection [21], and image segmentation [22], [23]. ViTs were also used for various tasks in satellite imagery, such as change detection [24] and deforestation monitoring [25]. The ViTs results were convincing, they even outperformed the classical convolutional architectures.…”

Section: A Vision Transformersmentioning

confidence: 99%

Fusion of Satellite Images and Weather Data With Transformer Networks for Downy Mildew Disease Detection

2023

View full text Add to dashboard Cite

Crop diseases significantly affect the quantity and quality of agricultural production. In a context where the goal of precision agriculture is to minimize or even avoid the use of pesticides, weather and remote sensing data with deep learning can play a pivotal role in detecting crop diseases, allowing localized treatment of crops. However, combining heterogeneous data such as weather and images remains a hot topic and challenging task. Recent developments in transformer architectures have shown the possibility of fusion of data from different domains, such as text-image. The current trend is to custom only one transformer to create a multimodal fusion model. Conversely, we propose a new approach to realize data fusion using three transformers. In this paper, we first solved the missing satellite images problem, by interpolating them with a ConvLSTM model. Then, we proposed a multimodal fusion architecture that jointly learns to process visual and weather information. The architecture is built from three main components, a Vision Transformer and two transformer-encoders, allowing to fuse both image and weather modalities. The results of the proposed method are promising achieving an overall accuracy of 97%.

show abstract

“…5(b). In this module, we design an attention mechanism based on the angle channel, while other attention mechanisms have gained special interest in the field of remote sensing in recent years [60,61]. The input of this module is the feature map extracted by the backbone network, and the output is a new feature map with the same shape as the input.…”

Section: Angle Channel Attention Modulementioning

confidence: 99%

Orientation-First Strategy With Angle Attention Module for Rotated Object Detection in Remote Sensing Images

Zhang

Wang

Zhang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recently, object detection in remote sensing images (RSIs) have received extensive attention and made significant progress. Nonetheless, the arbitrary orientations of objects in RSIs make their detection a challenging task. Most of the existing detection methods are difficult to extract the orientation features of objects due to the lack of directionality of conventional convolutions. In addition, the boundary discontinuity in angle regression affects the detection of object orientations. In response to these problems, this paper proposes an orientation-first refinement detector (OFRDet), which is based on a strategy that enables the detector to detect the angle of an object ahead of others and presets oriented anchors. In OFRDet, we propose an angle encoding regression module (AERM) and an angle channel attention module (ACAM). AERM transforms angle detection into multi-parameter regression, which eliminates boundary discontinuities. ACAM uses convolution kernels with different angles to extract directional features purposefully according to the preset oriented anchors. After these two modules, more accurate bounding boxes are generated and sent to the refined stage to obtain the final detection results. We evaluate our method and demonstrate the effectiveness of it by conducting experiments on two challenging and credible datasets, DOTA, HRSC2016. OFRDet achieves competitive results 79.56%, 96.29% mAP on the two datasets, respectively. Index Terms-Angle channel attention, angle encoding, remote sensing images, rotated object detection I. INTRODUCTION BJECT detection is a technique in computer vision that requires locating and identifying the certain object in the image. Remote sensing images (RSIs) are more challenging to be detected since the scale of RSIs is larger and the content is more complex than that of ordinary natural images [1]. In addition, objects are unevenly distributed on RSIs and are generally small. With the continuous development of deep learning technology, neural networks are widely used in image processing. Meanwhile the object detection based on convolutional neural networks (CNNs) have made great Manuscript

show abstract

A Vision Transformer Model for Convolution-Free Multilabel Classification of Satellite Imagery in Deforestation Monitoring

Cited by 55 publications

References 24 publications

Interpretable Deep Learning Framework for Land Use and Land Cover Classification in Remote Sensing Using SHAP

Interpretable Deep Learning Framework for Land Use and Land Cover Classification in Remote Sensing Using SHAP

Fusion of Satellite Images and Weather Data With Transformer Networks for Downy Mildew Disease Detection

Orientation-First Strategy With Angle Attention Module for Rotated Object Detection in Remote Sensing Images

Contact Info

Product

Resources

About