iDECODe: In-distribution Equivariance for Conformal Out-of-distribution Detection

Abstract

Machine learning methods such as deep neural networks (DNNs), despite their success across different domains, are known to often generate incorrect predictions with high confidence on inputs outside their training distribution. The deployment of DNNs in safety-critical domains requires detection of out-of-distribution (OOD) data so that DNNs can abstain from making predictions on those. A number of methods have been recently developed for OOD detection, but there is still room for improvement. We propose the new method iDECODe, leveraging in-distribution equivariance for conformal OOD detection. It relies on a novel base non-conformity measure and a new aggregation method, used in the inductive conformal anomaly detection framework, thereby guaranteeing a bounded false detection rate. We demonstrate the efficacy of iDECODe by experiments on image and audio datasets, obtaining state-of-the-art results. We also show that iDECODe can detect adversarial examples.

Conformal prediction for the design problem

“…Though originally based on the premise of exchangeable (e.g., independently and identically distributed) training and test data, the framework has since been generalized to handle various forms of distribution shift, including covariate shift [62,44], label shift [47], arbitrary distribution shifts in an online setting [20], and test distributions that are nearby the training distribution [15]. Conformal approaches have also been used to detect distribution shift [64,28,38,8,4,48,30].…”

PAC-Wrap: Semi-Supervised PAC Anomaly Detection

“…As discussed in [35], the mathematical structure of these methods is closely related to that of tolerance regions [18,32,38]. Inductive conformal anomaly detection [16,25] builds on ICP to guarantee a bounded false detection rate. In different literature, there are different terminology for the two user-specified inputs.…”