Continual Contrastive Learning for Cross-Dataset Scene Classification

Peng, Rui; Zhao, Wenzhi; Li, Kaiyuan; Ji, Fengcheng; Rong, Caixia

doi:10.3390/rs14205105

Cited by 5 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In CV, the combination of these two technologies is starting to be explored, and some works have already started to demonstrate how SSL methods are feasible to learn incrementally [17], [29], [30]. In EO, this combination has not yet been explored, to the best of authors' knowledge, apart from a few contributions combining weakly-supervision [31] and contrastive losses [32], [33] with CL. On the other hand, we observe an increasing interest in foundation models [15], [34], based also on new extensive datasets [14], [27], [28].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation

Marsocci

Scardapane

2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In the field of earth observation (EO), continual learning (CL) algorithms have been proposed to deal with large datasets by decomposing them into several subsets and processing them incrementally. The majority of these algorithms assume that data are, first, coming from a single source, and second, fully labeled. Real-world EO datasets are instead characterized by a large heterogeneity (e.g., coming from aerial, satellite, or drone scenarios), and for the most part they are unlabeled, meaning they can be fully exploited only through the emerging self-supervised learning (SSL) paradigm. For these reasons, in this article, we present a new algorithm for merging SSL and CL for remote sensing applications that we call continual Barlow twins. It combines the advantages of one of the simplest self-supervision techniques, i.e., Barlow twins, with the elastic weight consolidation method to avoid catastrophic forgetting. In addition, we evaluate the proposed continual SSL approach on a highly heterogeneous EO dataset, showing the effectiveness of this strategy on a novel combination of three almost non-overlapping domains datasets (airborne Potsdam, satellite US3D, and drone unmanned aerial vehicle semantic segmentation dataset), on a crucial downstream task in EO, i.e., semantic segmentation. Encouraging results show the superiority of SSL in this setting, and the effectiveness of creating an incremental effective pretrained feature extractor, based on ResNet50, without the need of relying on the complete availability of all the data, with a valuable saving of time and resources.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In [59], continual prototype calibration is proposed for few-shot classification CL. The authors in [32] and [33] make use of contrastive learning to learn effective representations that can reduce catastrophic forgetting. A fine-grained CL algorithm for SAR incremental target recognition is presented in [60].…”

Section: B Continual Learning In Eomentioning

confidence: 99%

Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation

Marsocci

Scardapane

2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…This significantly reduces the need for a large number of accurately labelled samples. For example, Peng Rui et al [46] used a contrast learning model and label propagation method to generate a large number of high-confidence labels in multi-scale unlabelled data, and finally used the expanded samples in a weakly supervised network to obtain the classification results of scene images. Liang et al [47] constructed a weakly supervised semantic segmentation network based on conditional generative adversarial networks and used a self-training method to generate pseudo-labels of unlabelled data by a generator to achieve weakly supervised semantic segmentation.…”

Section: Introductionmentioning

confidence: 99%

A Weak Sample Optimisation Method for Building Classification in a Semi-Supervised Deep Learning Framework

Wang,

Lin,

Huang

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

Deep learning has gained widespread interest in the task of building semantic segmentation modelling using remote sensing images; however, neural network models require a large number of training samples to achieve better classification performance, and the models are more sensitive to error patches in the training samples. The training samples obtained in semi-supervised classification methods need less reliable weakly labelled samples, but current semi-supervised classification research puts the generated weak samples directly into the model for applications, with less consideration of the impact of the accuracy and quality improvement of the weak samples on the subsequent model classification. Therefore, to address the problem of generating and optimising the quality of weak samples from training data in deep learning, this paper proposes a semi-supervised building classification framework. Firstly, based on the test results of the remote sensing image segmentation model and the unsupervised classification results of LiDAR point cloud data, this paper quickly generates weak image samples of buildings. Secondly, in order to improve the quality of the spots of the weak samples, an iterative optimisation strategy of the weak samples is proposed to compare and analyse the weak samples with the real samples and extract the accurate samples from the weak samples. Finally, the real samples, the weak samples, and the optimised weak samples are input into the semantic segmentation model of buildings for accuracy evaluation and analysis. The effectiveness of this paper’s approach was experimentally verified on two different building datasets, and the optimised weak samples improved by 1.9% and 0.6%, respectively, in the test accuracy mIoU compared to the initial weak samples. The results demonstrate that the semi-supervised classification framework proposed in this paper can be used to alleviate the model’s demand for a large number of real-labelled samples while improving the ability to utilise weak samples, and it can be used as an alternative to fully supervised classification methods in deep learning model applications that require a large number of training samples.

show abstract

“…Solutions dedicated to one area tend to fail the corner or generally less common observations. Thus, they better have long-term support [5]. The alternative can choose to be multi-functional and sufficiently capable of current concerns, meaning a successful algorithm should address its majors well and easily generalize to the rare rest [6].…”

Section: Introductionmentioning

confidence: 99%

Generalizing Spacecraft Recognition via Diversifying Few-Shot Datasets in a Joint Trained Likelihood

Yang,

Kong,

Lin

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

With the exploration of outer space, the number of space targets has increased dramatically, while the pressures of space situational awareness have also increased. Among them, spacecraft recognition is the foundation and a critical step in space situational awareness. However, unlike natural images that can be easily captured using low-cost devices, space targets can suffer from motion blurring, overexposure, and excessive dragging at the time of capture, which greatly affects the quality of the images and reduces the number of effective images. To this end, specialized or sufficiently versatile techniques are required, with dataset diversity playing a key role in enabling algorithms to categorize previously unseen spacecraft and perform multiple tasks. In this paper, we propose a joint dataset formulation to increase diversity. Our approach involves reformulating two local processes to condition the Conditional Neural Adaptive Processes, which results in global feature resampling schemes to adapt a pre-trained embedding function to be task-specific. Specifically, we employ variational resampling to category-wise auxiliary features, adding a generative constraint to amortize task-specific parameters. We also develop a neural process variational inference to encode representation, using grid density for conditioning. Our evaluation of the BUAA dataset shows promising results, with no-training performance close to a specifically designed learner and an accuracy rate of 98.2% on unseen categories during the joint training session. Further experiments on the Meta-dataset benchmark demonstrate at least a 4.6% out-of-distribution improvement compared to the baseline conditional models. Both dataset evaluations indicate the effectiveness of exploiting dataset diversity in few-shot feature adaptation. Our proposal offers a versatile solution for tasks across domains.

show abstract

Continual Contrastive Learning for Cross-Dataset Scene Classification

Cited by 5 publications

References 47 publications

Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation

Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation

A Weak Sample Optimisation Method for Building Classification in a Semi-Supervised Deep Learning Framework

Generalizing Spacecraft Recognition via Diversifying Few-Shot Datasets in a Joint Trained Likelihood

Contact Info

Product

Resources

About