Retrieving relative soft biometrics for semantic identification

Abstract

Abstract-Automatically describing pedestrians in surveillance footage is crucial to facilitate human accessible solutions for suspect identification. We aim to identify pedestrians based solely on human description, by automatically retrieving semantic attributes from surveillance images, alleviating exhaustive label annotation. This work unites a deep learning solution with relative soft biometric labels, to accurately retrieve more discriminative image attributes. We propose a Semantic Retrieval Convolutional Neural Network to investigate automatic retrieval of three soft biometric modalities, across a number of 'closed-world' and 'open-world' re-identification scenarios. Findings suggest that relative-continuous labels are more accurately predicted than absolute-binary and relative-binary labels, improving semantic identification in every scenario. Furthermore, we demonstrate a top rank-1 improvement of 23.2% and 26.3% over a traditional, baseline retrieval approach, in one-shot and multi-shot reidentification scenarios respectively.

Transfer Learning Based Approach for Semantic Person Retrieval

“…Figure 4 shows the CMC curve of each method and Table 1 shows the recognition rate at each rank. This shows our proposed approach achieves a higher recognition rate among the methods in related works and our proposed Method r=1 r = 5 r = 10 r = 20 r = 50 CR Avatar [ Method EER CR Avatar [1] 36.3 ETC [8] 43.3 SRCNN [7] 40.2 ResNet-152 [6] 30.9 Proposed 26.9 Proposed with FS 24.6 Table 2. EER of different methods for Task1 approach with feature selection (FS) achieves the highest recognition rate among the others.…”

“…Increasingly, deep learning is used for semantic person identification [6,7,8], by virtue of performance. [8] used the hand-crafted features and Extra Tree Classification (ETC) algorithm for identification while [7] used a Convolution Neural Network based approach named Semantic Retrieval Convolution Neural Network (SRCNN). These algorithms can identify 20.1% and 46.4% correct recognition rate at rank 1 for multi-shot identification respectively.…”

“…Using AVSS Challenge 2 Task 1 training dataset, we compare the semantic identification algorithm in related works with our proposed approach. The algorithms include Extra Tree Classifier (ETC) [8], SRCNN [7], fine-tuned ResNet [6] and Channel Representational (CR) Avatar Search [1]. For CR Avatar Search, we use the equation to calculate the similarity of the foreground to the template in [1] as match score as follows;…”

Semantic Face Signatures: Recognizing and Retrieving Faces by Verbal Descriptions

“…Furthermore, and to the best of our knowledge, no previous work has explored the semantic gap between humans and machines with respect to relative facial soft biometrics. Although some studies have addressed the retrieval of relative body [25] and clothing [10] attributes for subject identification, retrieval of relative facial attributes has only been studied for non-biometrics objectives [12], [26]. Therefore, the purpose of this paper is to explore subject retrieval using verbal descriptions or sample face images in a database of face images.…”

Soft Biometrics Classification Using Denoising Convolutional Autoencoders and Support Vector Machines

“…Whilst [4] estimated the age and gender of individuals from human faces in images using a CNN. Works by [5] and [6] presented architectures based on training the feature extraction part of a CNN (Convolution and Pooling layers) separately for each patch of an image and joining the features into a flattened vector that is fed to the Fully Connected layers of the network. Both approaches allow to perform multi-label classification.…”