Inductive Conformal Prediction: Theory and Application to Neural Networks

Papadopoulos, Harris

doi:10.5772/6078

Cited by 152 publications

(155 citation statements)

References 13 publications

Supporting

Mentioning

154

Contrasting

Order By: Relevance

“…On a number of real-world data sets, however, the predictive regions produced by GP-regression were no longer valid, i.e., they may become misleading when the correct prior is not known. The CP framework has been applied to classification using several popular learning algorithms, such as ANNs (Papadopoulos 2008), kNN (Nguyen and Luo 2012), SVMs (Devetyarov and Nouretdinov 2010;Makili et al 2011), decision trees (Johansson et al 2013a), random forests (Bhattacharyya 2011;Devetyarov and Nouretdinov 2010) and evolutionary algorithms (Johansson et al 2013b;Lambrou et al 2011). Although we in this study consider regression tasks, there is some overlap with previous studies on classification when it comes to design choices.…”

Section: Related Workmentioning

confidence: 99%

Regression conformal prediction with random forests

et al. 2014

View full text Add to dashboard Cite

Regression conformal prediction produces prediction intervals that are valid, i.e., the probability of excluding the correct target value is bounded by a predefined confidence level. The most important criterion when comparing conformal regressors is efficiency; the prediction intervals should be as tight (informative) as possible. In this study, the use of random forests as the underlying model for regression conformal prediction is investigated and compared to existing state-of-the-art techniques, which are based on neural networks and k-nearest neighbors. In addition to their robust predictive performance, random forests allow for determining the size of the prediction intervals by using out-of-bag estimates instead of requiring a separate calibration set. An extensive empirical investigation, using 33 publicly available data sets, was undertaken to compare the use of random forests to existing stateof-the-art conformal predictors. The results show that the suggested approach, on almost all confidence levels and using both standard and normalized nonconformity functions, produced significantly more efficient conformal predictors than the existing alternatives.

show abstract

Section: Related Workmentioning

confidence: 99%

Regression conformal prediction with random forests

et al. 2014

View full text Add to dashboard Cite

show abstract

“…At the aspect of the framework of CP, some modified models have been proposed to improve the flexibility of CP. In order to improve the computation efficiency of CP model, Papadopoulos proposed ICP (Inductive CP) for large data sets [14]. In our previous work, the HCCP (Hybrid-Compression CP) not only improves the computational efficiency but preserves the prediction efficiency as well [15].…”

Section: Conformal Predictor(cp)mentioning

confidence: 99%

Local Clustering Conformal Predictor for Imbalanced Data Classification

Wang

Chen

et al. 2013

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. The recently developed Conformal Predictor (CP) can provide calibrated confidence for prediction which is out of the traditional predictors' capacity. However, CP works for balanced data and fails in the case of imbalanced data. To handle this problem, Local Clustering Conformal Predictor (LCCP) which plugs a two-level partition into the framework of CP is proposed. In the first-level partition, the whole imbalanced training dataset is partitioned into some class-taxonomy data subsets. Secondly, the majority class examples proceed to be partitioned into some cluster-taxonomy data subsets by clustering method. To predict a new instance, LCCP selects the nearest cluster, incorporated with the minority class examples, to build a re-balanced training data. The designed LCCP model aims to not only provide valid confidence for prediction, but significantly improve the prediction efficiency as well. The experimental results show that LCCP model presents superiority than CP model for imbalanced data classification.

show abstract

“…There are two major categories of conformal predictors: transductive conformal predictors (TCP) [4] and inductive conformal predictors (ICP) [7]. In TCP, the nonconformity of an example (x i , y i ) is measured in relation to the multiset (x 1 , y 1 ), ..., (x k+1 ,ỹ), and predictions are made according to the following scheme:…”

Section: Conformal Predictionmentioning

confidence: 99%

Regression trees for streaming data with local performance guarantees

Johansson

Sönströd

Linusson

et al. 2014

2014 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Abstract-Online predictive modeling of streaming data is a key task for big data analytics. In this paper, a novel approach for efficient online learning of regression trees is proposed, which continuously updates, rather than retrains, the tree as more labeled data become available. A conformal predictor outputs prediction sets instead of point predictions; which for regression translates into prediction intervals. The key property of a conformal predictor is that it is always valid, i.e., the error rate, on novel data, is bounded by a preset significance level. Here, we suggest applying Mondrian conformal prediction on top of the resulting models, in order to obtain regression trees where not only the tree, but also each and every rule, corresponding to a path from the root node to a leaf, is valid. Using Mondrian conformal prediction, it becomes possible to analyze and explore the different rules separately, knowing that their accuracy, in the long run, will not be below the preset significance level. An empirical investigation, using 17 publicly available data sets, confirms that the resulting rules are independently valid, but also shows that the prediction intervals are smaller, on average, than when only the global model is required to be valid. All-in-all, the suggested method provides a data miner or a decision maker with highly informative predictive models of streaming data.

show abstract

Inductive Conformal Prediction: Theory and Application to Neural Networks

Cited by 152 publications

References 13 publications

Regression conformal prediction with random forests

Regression conformal prediction with random forests

Local Clustering Conformal Predictor for Imbalanced Data Classification

Regression trees for streaming data with local performance guarantees

Contact Info

Product

Resources

About