Uncertainty in Bayesian deep label distribution learning

Zheng, Rui; Zhang, Shulin; Liu, Lei; Luo, Yuhao; Sun, Mingzhai

doi:10.1016/j.asoc.2020.107046

Cited by 22 publications

(25 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by confident learning (Northcutt et al (2021)), we develop a matrix of label distribution with a Gaussian before estimating the uncertainty of the labels. For the data distribution of each label value, we use a Gaussian distribution to delimit the distribution rather than other multi-peaked distribution priors, and numerous kinds of literature have verified that this approach can eliminate the uncertainty (Liu et al (2021); Zheng et al (2021b); Ghosh et al (2021); Li et al (2022)). Furthermore, to capture the global correlation between labels to generate a standard label distribution, we employ a self-attention mechanism to model the label distribution matrix.…”

Section: Proposed Methodsmentioning

confidence: 99%

Label Distribution Learning via Implicit Distribution Representation

Zheng¹,

Jia²

2022

Preprint

View full text Add to dashboard Cite

In contrast to multi-label learning, label distribution learning characterizes the polysemy of examples by a label distribution to represent richer semantics. In the learning process of label distribution, the training data is collected mainly by manual annotation or label enhancement algorithms to generate label distribution. Unfortunately, the complexity of the manual annotation task or the inaccuracy of the label enhancement algorithm leads to noise and uncertainty in the label distribution training set. To alleviate this problem, we introduce the implicit distribution in the label distribution learning framework to characterize the uncertainty of each label value. Specifically, we use deep implicit representation learning to construct a label distribution matrix with Gaussian prior constraints, where each row component corresponds to the distribution estimate of each label value, and this row component is constrained by a prior Gaussian distribution to moderate the noise and uncertainty interference of the label distribution dataset. Finally, each row component of the label distribution matrix is transformed into a standard label distribution form by using the self-attention algorithm. In addition, some approaches with regularization characteristics are conducted in the training phase to improve the performance of the model.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Label Distribution Learning via Implicit Distribution Representation

Zheng¹,

Jia²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There are two different types of uncertainty in machine learning: epistemic uncertainty and aleatoric uncertainty [2,24,56].…”

Section: Uncertainty In Machine Learningmentioning

confidence: 99%

Exploiting epistemic uncertainty of the deep learning models to generate adversarial samples

Tuna

Çatak

Eskil

2022

Multimed Tools Appl

View full text Add to dashboard Cite

Deep neural network (DNN) architectures are considered to be robust to random perturbations. Nevertheless, it was shown that they could be severely vulnerable to slight but carefully crafted perturbations of the input, termed as adversarial samples. In recent years, numerous studies have been conducted in this new area called ``Adversarial Machine Learning” to devise new adversarial attacks and to defend against these attacks with more robust DNN architectures. However, most of the current research has concentrated on utilising model loss function to craft adversarial examples or to create robust models. This study explores the usage of quantified epistemic uncertainty obtained from Monte-Carlo Dropout Sampling for adversarial attack purposes by which we perturb the input to the shifted-domain regions where the model has not been trained on. We proposed new attack ideas by exploiting the difficulty of the target model to discriminate between samples drawn from original and shifted versions of the training data distribution by utilizing epistemic uncertainty of the model. Our results show that our proposed hybrid attack approach increases the attack success rates from 82.59% to 85.14%, 82.96% to 90.13% and 89.44% to 91.06% on MNIST Digit, MNIST Fashion and CIFAR-10 datasets, respectively.

show abstract

“…Distribution function for training data [39]. Some further tuning functions can be achieved through expert parameters [40]. Bernoulli distribution can be used for binominal atau 2-class polynominal labels.…”

Section: Bernoulli Distributionmentioning

confidence: 99%

Predictive Modeling Classification of Post-Flood and Abrasion Effects With Deep Learning Approach

Marleny¹,

Mambang

2022

TIERS

View full text Add to dashboard Cite

Floods and abrasion are the most common disasters in Indonesia. A lot of data is collected from post-flood and abrasion disasters. From the data released by BNPB, disaster data is directly based on the occurrence of disasters. From these data, we will test predictive modeling classification with a deep learning approach. Big data can be made through classification and predictive modeling. Our proposed model is a classification of predictive modeling of post-flood and abrasion data using the H2O deep learning approach. Deep learning H2O models can also be evaluated with specific model metrics, termination metrics, and performance charts. This approach is used to optimize the performance and accuracy of predictions during the modeling process using our dataset pool training. The big data to be processed will generate new knowledge for policies in decision making. Big data performance modeled with Deep Learning H2O is used to predict the Survival attributes of post-flood and abrasion sample datasets. The best metric performance is obtained from the maxout activation function with a 200-200 unit layer that gets an accuracy of 93.49% with AUC: 0.808 +/- 0.022 (micro average: 0.808).

show abstract

Uncertainty in Bayesian deep label distribution learning

Cited by 22 publications

References 37 publications

Label Distribution Learning via Implicit Distribution Representation

Label Distribution Learning via Implicit Distribution Representation

Exploiting epistemic uncertainty of the deep learning models to generate adversarial samples

Predictive Modeling Classification of Post-Flood and Abrasion Effects With Deep Learning Approach

Contact Info

Product

Resources

About