Improving image classification robustness using self‐supervision

Abstract

Self‐supervised learning allows training of neural networks without immense, high‐quality or labelled data sets. We demonstrate that self‐supervision furthermore improves robustness of models using small, imbalanced or incomplete data sets which pose severe difficulties to supervised models. For small data sets, the accuracy of our approach is up to 12.5% higher using MNIST and 15.2% using Fashion‐MNIST compared to random initialization. Moreover, self‐supervision influences the way of learning itself, which means that in case of small or strongly imbalanced data sets, it can be prevented that classes are not or insufficiently learned. Even if input data are corrupted and large image regions are missing from the training set, self‐supervision significantly improves classification accuracy (up to 7.3% for MNIST and 2.2% for Fashion‐MNIST). In addition, we analyse combinations of data manipulations and seek to generate a better understanding of how pretext accuracy and downstream accuracy are related. This is not only important to ensure optimal pretraining but also for training with unlabelled data in order to find an appropriate evaluation measure. As such, we make an important contribution to learning with realistic data sets and making machine learning accessible to application areas that require expensive and difficult data collection.

Geospatial Artificial Intelligence (GeoAI) in the Integrated Hydrological and Fluvial Systems Modeling: Review of Current Applications and Trends

“…Similarly, self-supervised image classification, particularly that developed in the robotic field [264], is rapidly being adopted in hydrological studies in, e.g., satellite image classification, fluvial landform classification, and landform change detection. Selfsupervised models use automatically generated pseudo-labels, significantly reducing manual labeling, one of the most time-consuming tasks in supervised classification [265]. Self-supervised image classification is enhanced by machine learning methods such as autoencoder and the generative adversarial network (GAN).…”