A Unifying Mutual Information View of Metric Learning: Cross-Entropy vs. Pairwise Losses

Boudiaf, Malik; Rony, Jérôme; Ziko, Imtiaz Masud; Granger, Éric; Pedersoli, Marco; Piantanida, Pablo; Ayed, Ismail Ben

doi:10.1007/978-3-030-58539-6_33

Cited by 94 publications

(99 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The RSA catalogue allowed us to contrast models on a completely unseen, expertly labelled set. First, the results show improvement on the state-of-the-art in galaxy classification [11] using deep metric learning techniques [16]. These results are evidenced by improved accuracy not equating to improved, learned representations.…”

Section: Introductionmentioning

confidence: 88%

“…This method requires large amounts of training samples similar to the ones expected when testing the model. On the other hand, metric learning proposes the idea of learning a complex, non-linear mapping [16]. This mapping maps high-dimensional input data to a lower-dimension manifold called an embedding space [16].…”

Section: E Deep Metric Learningmentioning

confidence: 99%

“…On the other hand, metric learning proposes the idea of learning a complex, non-linear mapping [16]. This mapping maps high-dimensional input data to a lower-dimension manifold called an embedding space [16]. The mapping typically uses pairwise distances (such as the Euclidean norm or cosine similarity) as a metric [16].…”

Section: E Deep Metric Learningmentioning

confidence: 99%

See 2 more Smart Citations

Transfer Learning and Deep Metric Learning for Automated Galaxy Morphology Representation

2022

View full text Add to dashboard Cite

Galaxy morphology characterisation is an important area of study, as the type and formation of galaxies offer insights into the origin and evolution of the universe. Owing to the increased availability of images of galaxies, scientists have turned to crowd-sourcing to automate the process of instance labelling. However, research has shown that using crowd-sourced labels for galaxy classification comes with many pitfalls. An alternative approach to galaxy classification is metric learning. Metric learning allows for improved representations for classification, anomaly detection, information retrieval, clustering and dimensionality reduction. Understanding the implications of this approach regarding crowd-sourced labels is of paramount importance if scientists intend to continue using them. This paper compares metric learning and classification models trained or fine-tuned on both the crowd-sourced Galaxy Zoo 2 (GZ2-H) dataset and expertly labelled EFIGI catalogue. The study uses the Revised Shapley-Ames (RSA) catalogue of bright galaxies, also labelled by experts, as an unseen test set. The RSA catalogue allows for an accurate comparison of the performance of the models at predicting the Hubble types of galaxies. The classification accuracy for the crowd-sourced and expert models indicated that the models are comparable on the surface. However, using alternative metrics, the results show that the models trained on the expert dataset outperformed the model trained on the crowd-sourced data in terms of actual vs predicted labels. Further, the results show that fine-tuning a model pre-trained on crowd-sourced data can outperform the state-of-the-art in galaxy characterisation. The models trained to predict the Hubble types of galaxies are better when fine-tuned using the Proxy-NCA and Normalised-Softmax loss functions than with other pairwise losses. The Normalised-Softmax loss yielded the best overall 9-class models with accuracies at 30.88% (GZ2-H) and 30.05% (EFIGI) and MAP values of 0.3483 (GZ2-H) and 0.3889. The Proxy-NCA loss produced the second-best overall 9-class models with accuracies at 30.33% (GZ2-H) and 20.03% (EFIGI) and MAP values of 0.3577 (GZ2-H) and 0.3917 (EFIGI). Finally, the paper highlights the need for caution when utilising crowd-sourced labels; however, it argues that transfer learning from crowd-sourced labelled data to expert-labelled data can still lead to significant improvements.

show abstract

Section: Introductionmentioning

confidence: 88%

Section: E Deep Metric Learningmentioning

confidence: 99%

See 1 more Smart Citation

Transfer Learning and Deep Metric Learning for Automated Galaxy Morphology Representation

2022

View full text Add to dashboard Cite

show abstract

“…Here we present some of the basic objectives used in visual transfer learning using KG, which can be augmented with additional regularization terms or hyperparameters. Although work [13,73] showed that the objectives have a smaller impact on the learned DNN than suspected, there are configurations of visual and semantic embedding space that only allow certain objectives to be applied. We define l ∈ R K as the network's output (logits) vector, and t ∈ 0, 1 K as the one-hot encoded vector of targets, where t 1 = 1.…”

Section: Training Objectives For Joint Embeddingsmentioning

confidence: 99%

A survey on visual transfer learning using knowledge graphs

Monka

Halilaj

Rettinger

2022

View full text Add to dashboard Cite

The information perceived via visual observations of real-world phenomena is unstructured and complex. Computer vision (CV) is the field of research that attempts to make use of that information. Recent approaches of CV utilize deep learning (DL) methods as they perform quite well if training and testing domains follow the same underlying data distribution. However, it has been shown that minor variations in the images that occur when these methods are used in the real world can lead to unpredictable and catastrophic errors. Transfer learning is the area of machine learning that tries to prevent these errors. Especially, approaches that augment image data using auxiliary knowledge encoded in language embeddings or knowledge graphs (KGs) have achieved promising results in recent years. This survey focuses on visual transfer learning approaches using KGs, as we believe that KGs are well suited to store and represent any kind of auxiliary knowledge. KGs can represent auxiliary knowledge either in an underlying graph-structured schema or in a vector-based knowledge graph embedding. Intending to enable the reader to solve visual transfer learning problems with the help of specific KG-DL configurations we start with a description of relevant modeling structures of a KG of various expressions, such as directed labeled graphs, hypergraphs, and hyper-relational graphs. We explain the notion of feature extractor, while specifically referring to visual and semantic features. We provide a broad overview of knowledge graph embedding methods and describe several joint training objectives suitable to combine them with high dimensional visual embeddings. The main section introduces four different categories on how a KG can be combined with a DL pipeline: 1) Knowledge Graph as a Reviewer; 2) Knowledge Graph as a Trainee; 3) Knowledge Graph as a Trainer; and 4) Knowledge Graph as a Peer. To help researchers find meaningful evaluation benchmarks, we provide an overview of generic KGs and a set of image processing datasets and benchmarks that include various types of auxiliary knowledge. Last, we summarize related surveys and give an outlook about challenges and open issues for future research.

show abstract

“…Compared with the variance loss function, the problem of updating weights and bias too slowly was overcome by this method. The updating of weights and deviations is affected by errors [37,38]. For this reason, when the error is large enough, the updating speed of weights is very fast.…”

Section: Cross Entropymentioning

confidence: 99%

A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects

Gao

et al. 2021

Forests

View full text Add to dashboard Cite

In recent years, due to the shortage of timber resources, it has become necessary to reduce the excessive consumption of forest resources. Non-destructive testing technology can quickly find wood defects and effectively improve wood utilization. Deep learning has achieved significant results as one of the most commonly used methods in the detection of wood knots. However, compared with convolutional neural networks in other fields, the depth of deep learning models for the detection of wood knots is still very shallow. This is because the number of samples marked in the wood detection is too small, which limits the accuracy of the final prediction of the results. In this paper, ResNet-34 is combined with transfer learning, and a new TL-ResNet34 deep learning model with 35 convolution depths is proposed to detect wood knot defects. Among them, ResNet-34 is used as a feature extractor for wood knot defects. At the same time, a new method TL-ResNet34 is proposed, which combines ResNet-34 with transfer learning. After that, the wood knot defect dataset was applied to TL-ResNet34 for testing. The results show that the detection accuracy of the dataset trained by TL-ResNet34 is significantly higher than that of other methods. This shows that the final prediction accuracy of the detection of wood knot defects can be improved by TL-ResNet34.

show abstract

A Unifying Mutual Information View of Metric Learning: Cross-Entropy vs. Pairwise Losses

Cited by 94 publications

References 29 publications

Transfer Learning and Deep Metric Learning for Automated Galaxy Morphology Representation

Transfer Learning and Deep Metric Learning for Automated Galaxy Morphology Representation

A survey on visual transfer learning using knowledge graphs

A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects

Contact Info

Product

Resources

About