Multi-stage classifier design

Abstract

In many classification systems, sensing modalities have different acquisition costs. It is often unnecessary to use every modality to classify a majority of examples. We study a multi-stage system in a prediction time cost reduction setting, where the full data is available for training, but for a test example, measurements in a new modality can be acquired at each stage for an additional cost. We seek decision rules to reduce the average measurement acquisition cost. We formulate an empirical risk minimization problem (ERM) for a multi-stage reject classifier, wherein the stage k classifier either classifies a sample using only the measurements acquired so far or rejects it to the next stage where more attributes can be acquired for a cost. To solve the ERM problem, we factorize the cost function into classification and rejection decisions. We then transform reject decisions into a binary classification problem. We construct stage-by-stage global surrogate risk, develop an iterative algorithm in the boosting framework and present convergence results. We test our work on synthetic, medical and explosives detection datasets. Our results demonstrate that substantial cost reduction without a significant sacrifice in accuracy is achievable.

Adversarial Autoencoder and Multi-Task Semi-Supervised Learning for Multi-stage Process

“…Our method builds on three subfields of machine learning, namely data imputation [3], multi-task learning [20] and multi-stage or decision cascades [15,17]. Data Imputation with Autoencoders -Many methods for data imputation have been proposed, ranging from simple column average to complex imputations based on statistical and machine learning models.…”

“…In contrast, multi-stage classifiers shown in Fig. 1(b), can deal with multi-class problems and can make classification decisions at any stage [15]. The approach proposed in [12] explores the connection between deep models and multi-stage classifiers such that classifiers can be jointly optimized and they can cooperate across the stages.…”

“…In many applications including network intrusion detection [15] and medical diagnosis [1], decision systems are composed of an ordered sequence of stages, which can be referred to as a multi-stage process. For selection processes (such as hiring or student intake), for instance, the applicants submit general information in the initial stages, such as resumes.…”

Feature-Sharing in Cascade Detection Systems With Multiple Applications

“…Trapeznikov et al studied a generalization of the cascade that was termed multi-stage sequential reject classifier (MSRC), which is simply the cascade with an additional positive decision [19] or multiple additional (classification) decisions [20] at intermediate stages. Their resource-consumption model is 'nebulous', i.e.…”

Cascade classification for diagnosing dementia

“…The first stage is trained using the entire dataset, while the following stages are trained using specific regions of interest. Cascade classifiers have been widely used in many applications, such as in hand writing recognition, face recognition and medical diagnoses [25,26,27].…”