Conformal Predictors (CPs) are machine learning algorithms that can provide predictions complemented with valid confidence measures. In medical diagnosis, such measures are highly desirable, as medical experts can gain additional information for each machine diagnosis. A risk assessment in each prediction can play an important role for medical decision making, in which the outcome can be critical for the patients. Several classical machine learning methods can be incorporated into the CP framework. In this paper, we propose a CP that makes use of evolved rule sets generated by a genetic algorithm (GA). The rule-based GA has the advantage of being human readable. We apply our method on two real-world datasets for medical diagnosis, one dataset for breast cancer diagnosis, which contains data gathered from fine needle aspirate of breast mass; and one dataset for ovarian cancer diagnosis, which contains proteomic patterns identified in serum. Our results on both datasets show that the proposed method is as accurate as the classical techniques, while it provides reliable and useful confidence measures.

Abstract. Venn Predictors (VPs) are machine learning algorithms that can provide well calibrated multiprobability outputs for their predictions. The only drawback of Venn Predictors is their computational inefficiency, especially in the case of large datasets. In this work, we propose an Inductive Venn Predictor (IVP) which overcomes the computational inefficiency problem of the original Venn Prediction framework. Each VP is defined by a taxonomy which separates the data into categories. We develop an IVP with a taxonomy derived from a multiclass Support Vector Machine (SVM), and we compare our method with other probabilistic methods for SVMs, namely Platt's method, SVM Binning, and SVM with Isotonic Regression. We show that these methods do not always provide well calibrated outputs, while our IVP will always guarantee this property under the i.i.d. assumption.

No abstract