Transfer Learning Based Approach for Semantic Person Retrieval

Abstract

Many algorithms for semantic person retrieval suffer from a lack of training data often due to the difficulties in constructing a large dataset. We therefore propose a transfer learning based approach for semantic person identification and semantic person search. We apply the fine-tuned Mask R-CNN and DenseNet-161 for detection and attribute classification. The networks were pre-trained on the MS COCO and ILSVRC 2012 datasets. Our proposed approach achieves the highest recognition rate at each rank of CMC curve for semantic person identification and the highest average localization precision for semantic person search on our validation dataset.

Visual appearance based person retrieval in unconstrained environment videos

“…This section discusses qualitative and quantitative results derived from the 41 test sequences. The proposed algorithm results are also compared with the AVSS 2018 challenge II participants [14][15][16] results. The performance of the proposed algorithm is evaluated using the True Positive Rate (TPR) and IoU.…”

“…The proposed approach and the AVSS 2018 challenge II approaches [7,[14][15][16] are compared to evaluate the performance. Table 3 compares average IoU and IoU ≥ 0.4 as an evaluation metrics.…”

“…Deep learning based methodologies are widely used for semantic description based person retrieval due to its efficient learning. Convolutional Neural Network (CNN) based algorithms are reported for person attribute recognition [10][11][12] as well as for person retrieval [5,[13][14][15][16]. Semantic Retrieval Convolution Neural Network (SRCNN) [10] retrieves the soft biometrics with recognition rate of 20.1% and 46.4% at rank-1 for one-shot and multi-shot identification respectively on SoBiR [8] dataset.…”

“…Moreover, [15] uses AlexNet [25] which has lower retrieval accuracy compared to deeper networks like ResNet [26] and DenseNet [27]. The transfer learning based approach by Yaguchi et al [14] also uses Mask R-CNN for person detection and DenseNet-161 for soft biometric classification. The Hamming loss is used to calculate matching score of the semantic query.…”

“…The paper uses one more model for pose estimation while extracting the patch for color and texture. In the AVSS challenge II, [14] achieved state-of-the-art per formance with an average IoU of 0.51 and [16] had 75.9% of frames with an IoU greater than 0.4. The use of tracking in the video sequences provides superior detection [16] .…”