Eagle-Eyed Multitask CNNs for Aerial Image Retrieval and Scene Classification

Liu, Yishu; Han, Zhengzhuo; Chen, Conghui; Ding, Lei

doi:10.1109/tgrs.2020.2979011

Cited by 25 publications

(12 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous research on RSIR have ignored the advantages of joint optimization of RSIR and scene classification. To overcome this limitation, Liu et al have presented an eagle-eyed multitask CNN integrating three tasks, i.e., center-metric learning, similarity distribution learning, and aerial scene classification in a network [86]. The extensive experiments over four public aerial image sets demonstrate its better performance than all of the existing methods.…”

Section: ) Metric Learning-based Methodsmentioning

confidence: 99%

Remote Sensing Image Retrieval in the Past Decade: Achievements, Challenges, and Future Directions

Zhou

Guan

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image retrieval (RSIR) aims to search and retrieve the images of interest from a large remote sensing (RS) image archive, which has remained to be a hot topic over the past decade. Benefited from the advent and progress of deep learning, RSIR has been promoted by developing novel approaches, constructing new datasets, and exploring potential applications. To the best of our knowledge, there lacks a comprehensive review of RSIR achievements, including systematic and hierarchical categorization of RSIR methods and benchmark datasets in the past decade. This article therefore provides a systematic survey of the recently published RSIR methods and benchmarks by reviewing more than 200 papers. To be specific, in terms of image source, label, and modality, we first group the RSIR methods into some hierarchical categories, each of which is reviewed in detail. Following the categorization of the RSIR methods, we list the benchmark datasets publically available for performance evaluation, and present our newly collected RSIR dataset. Moreover, some of the existing RSIR methods are selected and evaluated on the representative benchmark datasets. The results demonstrate that deep learning-based methods are currently the dominant RSIR approaches and outperform handcrafted feature-based methods by a significant margin. Finally, we discuss the main challenges of RSIR, and point out some potential directions for the future RSIR research.

show abstract

Section: ) Metric Learning-based Methodsmentioning

confidence: 99%

Remote Sensing Image Retrieval in the Past Decade: Achievements, Challenges, and Future Directions

Zhou

Guan

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In [48], a wide-context attention network is introduced to learn the correlation of local descriptors with wide context information by employing channel dependence-and spatial context-attention modules. In [38], a center-metric learning method, which employs the positive-negative center loss function for modeling metric space, is proposed to characterize within-class variations. In [49], a discriminative distillation network is introduced to increase the interclass variations and to reduce the intraclass differences.…”

Section: B Multi-task-driven Cbir Methodsmentioning

confidence: 99%

“…Accordingly, few DL-based multi-task learning (MTL) methods have been recently introduced in RS for CBIR applications. As an example, in [38], RS image similarity learning based on triplet loss is combined with the This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ scene classification task.…”

Section: Introductionmentioning

confidence: 99%

Plasticity-Stability Preserving Multi-Task Learning for Remote Sensing Image Retrieval

Sümbül

Demir

2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Deep learning-based multi-task learning (MTL) methods have recently attracted attention for content-based image retrieval (CBIR) applications in remote sensing (RS). For a given set of tasks (e.g., scene classification, semantic segmentation, and image reconstruction), existing MTL methods employ a joint optimization algorithm on the direct aggregation of task-specific loss functions. Such an approach may provide limited CBIR performance when: 1) tasks compete or even distract each other; 2) one of the tasks dominates the whole learning procedure; or 3) characterization of each task is underperformed compared to single-task learning. This is mainly due to the lack of: 1) plasticity condition (which is associated with sensitivity to new information) or 2) stability condition (which is associated with protection from radical disruptions by new information) of the whole learning procedure. To avoid this issue, as a first time, we propose a novel plasticity-stability preserving MTL (PLASTA-MTL) approach to ensure the plasticity and the stability conditions of the whole learning procedure independently of the number and type of tasks. This is achieved by defining two novel loss functions. The first loss function is the plasticity preserving loss (PPL) function that aims to enforce the global image representation space to be sensitive to new information learned with each task. This is achieved by minimizing the difference of gradient magnitudes for the global representation and task-specific embedding spaces. The second loss function is the stability preserving loss (SPL) function that aims to protect the global representation space radically disrupted by a new task. This is achieved by minimizing the angular distances between the task gradients over global representation space. To effectively employ the proposed loss functions, we also introduce a novel sequential optimization algorithm. Experimental results show the effectiveness of the proposed approach compared to the state-of-the-art MTL methods in the context of CBIR.

show abstract

“…Adressing the specificity of remote sensing images, [24] designed an attention module focusing on objects typically found in remote sensing images to boost scene classification performance. [18] designed a discriminative training loss taking into account the high intraclass variation.…”

Section: Remote Sensingmentioning

confidence: 99%

Unifying Remote Sensing Image Retrieval and Classification with Robust Fine-tuning

Gominski¹,

Gouet-Brunet²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

Advances in high resolution remote sensing image analysis are currently hampered by the difficulty of gathering enough annotated data for training deep learning methods, giving rise to a variety of small datasets and associated dataset-specific methods. Moreover, typical tasks such as classification and retrieval lack a systematic evaluation on standard benchmarks and training datasets, which make it hard to identify durable and generalizable scientific contributions. We aim at unifying remote sensing image retrieval and classification with a new large-scale training and testing dataset, SF300 1 , including both vertical and oblique aerial images and made available to the research community, and an associated finetuning method. We additionally propose a new adversarial fine-tuning method for global descriptors. We show that our framework systematically achieves a boost of retrieval and classification performance on nine different datasets compared to an ImageNet pretrained baseline, with currently no other method to compare to.

show abstract

Eagle-Eyed Multitask CNNs for Aerial Image Retrieval and Scene Classification

Cited by 25 publications

References 71 publications

Remote Sensing Image Retrieval in the Past Decade: Achievements, Challenges, and Future Directions

Remote Sensing Image Retrieval in the Past Decade: Achievements, Challenges, and Future Directions

Plasticity-Stability Preserving Multi-Task Learning for Remote Sensing Image Retrieval

Unifying Remote Sensing Image Retrieval and Classification with Robust Fine-tuning

Contact Info

Product

Resources

About