Instance weighting through data imprecisiation

Lienen, Julian; Hüllermeier, Eyke

doi:10.1016/j.ijar.2021.04.002

Cited by 8 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, from a data modeling perspective, it is important to recognize that the imprecisiation of the target information pays off with higher validity, i.e., it is more likely that the true label is covered by the target labels. This is completely in line with the idea of data imprecisiation in the context of so-called superset learning (Lienen and Hüllermeier 2021b). Thus, ambiguating presumably corrupt training information appears to be a useful means to counter the influence of mislabeling.…”

Section: Credal Labelingsupporting

confidence: 82%

Mitigating Label Noise through Data Ambiguation

Lienen,

Hüllermeier

2024

AAAI

View full text Add to dashboard Cite

Label noise poses an important challenge in machine learning, especially in deep learning, in which large models with high expressive power dominate the field. Models of that kind are prone to memorizing incorrect labels, thereby harming generalization performance. Many methods have been proposed to address this problem, including robust loss functions and more complex label correction approaches. Robust loss functions are appealing due to their simplicity, but typically lack flexibility, while label correction usually adds substantial complexity to the training setup. In this paper, we suggest to address the shortcomings of both methodologies by "ambiguating" the target information, adding additional, complementary candidate labels in case the learner is not sufficiently convinced of the observed training label. More precisely, we leverage the framework of so-called superset learning to construct set-valued targets based on a confidence threshold, which deliver imprecise yet more reliable beliefs about the ground-truth, effectively helping the learner to suppress the memorization effect. In an extensive empirical evaluation, our method demonstrates favorable learning behavior on synthetic and real-world noise, confirming the effectiveness in detecting and correcting erroneous training labels.

show abstract

Section: Credal Labelingsupporting

confidence: 82%

Mitigating Label Noise through Data Ambiguation

Lienen,

Hüllermeier

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…By contrast, data imprecisiation [24,34] refers to soft computing approaches by which data affected by some form of uncertainty are transformed into imprecise (soft) observations, that is distributions over possible instances, which are then used to train specialized ML algorithms. Formally speaking, an imprecisiation scheme is a function is : X × Y → [0, 1] X×Y , where X is the feature space.…”

Section: Data Augmentation and Imprecisiation Methods To Manage Insta...mentioning

confidence: 99%

“…More in particular, to provide a more self-contained and detailed discussion, we will focus our experimental analysis on a specific setting, the medical one, which is of particular relevance due to its critical characteristics as well as due to it being one of the fields of applications of ML in which the problem of IV has been more frequently acknowledged. Finally, the last part of the paper will aim to build on the rubble left by the first part, and it will focus on the hypothesis whether more advanced learning and regularization methods (grounding on, either, data augmentation [33] or data imprecisiation [34]) will achieve increased robustness in face of the same perturbations (H 2 ). To address these two research questions, and motivated by the lack of datasets that rep-resent and allow to investigate this complex form of uncertainty, we will rely on a large gold-standard medical dataset that had been proposed for the task of COVID-19 diagnosis, a major impactful concern, which was specifically constructed with the help of clinical laboratory medicine to study IV, grounding on previous knowledge in this domain [30,35,36].…”

Section: Introductionmentioning

confidence: 99%

Everything is varied: The surprising impact of instantial variation on ML reliability

Campagner

Famiglini

Carobene

et al. 2023

Applied Soft Computing

View full text Add to dashboard Cite

“…Such methods can be found in a wide range of domains, including natural language processing [22] and computer vision [21,27,56,68]. As self-training can be considered as a general learning paradigm, where a model suggests itself labels to learn from, it has been wrapped around various model types, e.g., support vector machines [44], decision trees [62] and most prominently with neural networks [50]. Notably, it lays the foundation for so-called distillation models, e.g., in self-distillation [35] or studentteacher setups [53,68].…”

Section: Related Workmentioning

confidence: 99%

Conformal Credal Self-Supervised Learning

Lienen¹,

Demir²,

Hüllermeier³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In semi-supervised learning, the paradigm of self-training refers to the idea of learning from pseudo-labels suggested by the learner itself. Across various domains, corresponding methods have proven effective and achieve state-of-the-art performance. However, pseudo-labels typically stem from ad-hoc heuristics, relying on the quality of the predictions though without guaranteeing their validity. One such method, so-called credal self-supervised learning, maintains pseudo-supervision in the form of sets of (instead of single) probability distributions over labels, thereby allowing for a flexible yet uncertainty-aware labeling. Again, however, there is no justification beyond empirical effectiveness. To address this deficiency, we make use of conformal prediction, an approach that comes with guarantees on the validity of set-valued predictions. As a result, the construction of credal sets of labels is supported by a rigorous theoretical foundation, leading to better calibrated and less error-prone supervision for unlabeled data. Along with this, we present effective algorithms for learning from credal self-supervision. An empirical study demonstrates excellent calibration properties of the pseudo-supervision, as well as the competitiveness of our method on several benchmark datasets.Preprint. Under review.

show abstract

Instance weighting through data imprecisiation

Cited by 8 publications

References 17 publications

Mitigating Label Noise through Data Ambiguation

Mitigating Label Noise through Data Ambiguation

Everything is varied: The surprising impact of instantial variation on ML reliability

Conformal Credal Self-Supervised Learning

Contact Info

Product

Resources

About