Reliability Maps: A Tool to Enhance Probability Estimates and Improve Classification Accuracy

Kull, Meelis; Flach, Peter A.

doi:10.1007/978-3-662-44851-9_2

Cited by 15 publications

(13 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This means, that equivalent to classification tasks on unbalanced data sets, the uncertainty associated with single samples or small groups of samples may potentially get biased towards the performance on the rest of the data set. But for practical applications, assessing the reliability of a predicted confidence would give much more possibilities than an aggregated reliability based on some testing data, which are independent from the current situation [341].…”

Section: A Conclusion -How Well Do the Current Uncertainty Quantifica...mentioning

confidence: 99%

A Survey of Uncertainty in Deep Neural Networks

Gawlikowski¹,

Tassi²,

Ali³

et al. 2021

Preprint

163

158

View full text Add to dashboard Cite

and safety-critical real world applications remains limited. The main factors responsible for this limitation are• the lack of expressiveness and transparency of a deep neural network's inference model, which makes it difficult to trust their outcomes [2],• the inability to distinguish between in-domain and out-ofdomain samples [11], [12] and the sensitivity to domain shifts [13],• the inability to provide reliable uncertainty estimates for a deep neural network's decision [14] and frequently occurring overconfident predictions [15], [16],• the sensitivity to adversarial attacks that make deep neural networks vulnerable for sabotage [17], [18], [19].

show abstract

Section: A Conclusion -How Well Do the Current Uncertainty Quantifica...mentioning

confidence: 99%

A Survey of Uncertainty in Deep Neural Networks

Gawlikowski¹,

Tassi²,

Ali³

et al. 2021

Preprint

163

158

View full text Add to dashboard Cite

show abstract

“…It is known however that the likelihood probabilities produced by classification methods are not always reliable. Methods for estimating the reliability of such likelihood probabilities have been proposed in the machine learning literature . A possible enhancement of the proposed approach would be to integrate heuristics that take into account such reliability estimates.…”

Section: Discussionmentioning

confidence: 99%

Predicting process performance: A white‐box approach based on process models

Verenich

Dumas

Rosa

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Predictive business process monitoring methods exploit historical process execution logs to provide predictions about running instances of a process. These predictions enable process workers and managers to preempt performance issues or compliance violations. A number of approaches have been proposed to predict quantitative process performance indicators for running instances of a process, including remaining cycle time, cost, or probability of deadline violation. However, these approaches adopt a black-box approach, insofar as they predict a single scalar value without decomposing this prediction into more elementary components. In this paper, we propose a white-box approach to predict performance indicators of running process instances. The key idea is to first predict the performance indicator at the level of activities and then to aggregate these predictions at the level of a process instance by means of flow analysis techniques. The paper develops this idea in the context of predicting the remaining cycle time of ongoing process instances. The proposed approach has been evaluated on real-life event logs and compared against several baselines. KEYWORDS explainable artificial intelligence, flow analysis, predictive process monitoring, process mining, transparent models INTRODUCTIONPredictive business process monitoring techniques seek to predict the future state or properties of ongoing executions of a process based on models extracted from historical event logs. A wide range of predictive business process monitoring techniques have been proposed to predict, for example, compliance violations, 1,2 the next activity or the remaining sequence of activities of a process instance, 3,4 or quantitative process performance indicators, such the remaining cycle time of a process instance. 5-7 These predictions can be used to alert process workers to problematic process instances or to support resource allocation decisions, eg, to allocate additional resources to instances that are at risk of a deadline violation.This article addresses the problem of predicting quantitative process performance indicators, with a specific focus on predicting the remaining cycle time of ongoing process instances. Existing approaches to this problem adopt a "black-box" approach by building stochastic models or regression models which, given a process instance, predict the remaining execution time as a single scalar value, without seeking to explain this prediction in terms of more elementary components. Yet, quantitative performance indicators such as cost or time are aggregations of corresponding performance indicators of the activities composing the process. For example, the cycle time of a process instance with sequentially performed activities consists of the sum of the cycle time of the activities performed in that process instance. In this respect, existing known as flow analysis. 8 The idea of flow analysis is to estimate a quantitative performance indicator at the level of a process by aggregating the estimated values of this perfo...

show abstract

“…Another example is that, in medical diagnosis, a patient is more interested in the reliability of the test results for his/her own case instead of the reliability of the test on average 270 . Therefore, the necessary action for machine learning used in safety‐related engineering problems is to customize application‐specific loss functions 271 to assess each instance's risk instead of aggregate reliability 272 …”

Section: Conclusion and Potential Research Directionsmentioning

confidence: 99%

Fatigue modeling using neural networks: A comprehensive review

Chen

Liu

2022

Fatigue Fract Eng Mat Struct

135

View full text Add to dashboard Cite

Neural network (NN) models have significantly impacted fatigue‐related engineering communities and are expected to increase rapidly due to the recent advancements in machine learning and artificial intelligence. A comprehensive review of fatigue modeling methods using NNs is lacking and will help to recognize past achievements and suggest future research directions. Thus, this paper presents a survey of 251 publications between 1990 and July 2021. The NN modeling in fatigue is classified into five applications: fatigue life prediction, fatigue crack, fatigue damage diagnosis, fatigue strength, and fatigue load. A wide range of NN architectures are employed in the literature and are summarized in this review. An overview of important considerations and current limitations for the application of NNs in fatigue is provided. Statistical analysis for the past and the current trend is provided with representative examples. Existing gaps and future research directions are also presented based on the reviewed articles.

show abstract

Reliability Maps: A Tool to Enhance Probability Estimates and Improve Classification Accuracy

Cited by 15 publications

References 14 publications

A Survey of Uncertainty in Deep Neural Networks

A Survey of Uncertainty in Deep Neural Networks

Predicting process performance: A white‐box approach based on process models

Fatigue modeling using neural networks: A comprehensive review

Contact Info

Product

Resources

About