Classifier Ensembles with Trajectory Under-Sampling for Face Re-Identification

Abstract

Abstract:In person re-identification applications, an individual of interest may be covertly tracked and recognized based on trajectories of faces or other distinguishing information captured with video surveillance camera. However, a varying level of imbalance often exists between target and non-target facial captures, and this imbalance level may differ from what was considered during design. The performance of face classification systems typically declines in such cases because, to avoid bias towards the majority class (non-target), they tend to optimize the overall accuracy under a balance class assumption. Specialized classifier ensembles trained on balanced data, where non-target samples are selected through random under-sampling or cluster-based sampling, have been proposed in literature, but they suffer from loss of information and low diversity and accuracy. In this paper, a new ensemble method is proposed for generating a diverse pool of classifiers, each one trained on different levels of class imbalance and complexity for a greater diversity of opinion. Ensembles with Trajectory Under Sampling (EoC-TUS) allows to select subsets of non-target training data based on trajectories information. Variants of these ensembles can give more importance to the most efficient classifiers in identifying target samples, or define efficient and diverse decision boundaries by starting selection of trajectories from the farthest ones to the target class. For validation, experiments are conducted using videos captured in the Faces In Action dataset, and compared to several baseline techniques. The proposed EoC-TUS outperforms state-of-the-art techniques in terms of accuracy and diversity over a range of imbalance levels in the input video.

F-measure curves: A tool to visualize classifier performance under imbalance

“…Non-target faces captured in videos under various challenging conditions are compared to those of the target individual using a video-to-video face recognition system. One important challenge in this application is that the number of faces captured from the target individual (positive class) is typically limited and greatly outnumbered by non-target ones (negative class) [31,32,33,34]. We use the COX Face dataset [35] that is a dataset for face recognition in video surveillance [35] and contains videos from 1000 participants captured with 4 cameras under different capture conditions.…”

Progressive boosting for class imbalance and its application to face re-identification

“…In contrast, random under-sampling (RUS) (Seiffert et al, 2010) is more computationally efficient, but suffers from information loss. Partitional approaches (Soleymani et al, 2016a;Yan et al, 2003;Li et al, 2013) avoid information loss by splitting the negative class to uncorrelated subsets and training classifiers using all of these subsets.…”

“…However, some potentially informative samples may be overlooked from these subsets in under-sampling pro-260 cess. In partitional approaches (Soleymani et al, 2016a;Yan et al, 2003;Li et al, 2013) bootstraps are selected without replacement either randomly (Yan et al, 2003), by clustering (Li et al, 2013) or based on a prior knowledge from the application (like trajectories in video surveillance applications such as face 265 re-identification (Soleymani et al, 2016a)). In these ensemble bootstraps are drawn from a set of negative samples that reduces size in each iteration.…”

“…The contribution of the classifiers in the ensemble are then weighted based on the distance between the corresponding negative cluster and positive class. In (Soleymani et al, 2016a), partitioning negative class is done by selecting samples 280 from a set of trajectories that are formed based on the tracking information, as found in several video surveillance applications like face re-identification. In this approach, data from the trajectories are accumulated as the training iteration proceeds and therefore, base classifiers in the ensemble are trained on dif-285 ferent imbalance levels to increase robustness of the ensemble to the possible variations in the skew level and complexity of operational data.…”

“…The level of imbalance may be difficult to estimate accurately during operations and a static approach that accounts for variations in imbalance level of data may be of more interest. In the static approach of (Soleymani et al, 2016a), all the face captures are regrouped into trajectories. The classifiers are trained 380 using a positive class trajectory and different numbers of negative class trajectories to design diverse and accurate classifiers.…”