Person Re-Identification on a Mobile Robot Using a Depth Camera

Abstract

In this paper, we designed and implemented a realtime person re-identification API on a mobile robot, for a closedand open-world setting, using only the IR gray value image of a depth camera. Since common datasets are not usable we created our own dataset using the IR gray value images, the pose and image processing techniques. Then we trained the state-of-theart neural network for person re-identification with common parameters and methods. For running it in real-time, we sped up the model as well as the application. It is possible to reidentify three persons at once, at around 10 FPS. Our model reaches as closed-world setting a rank-1-accuracy of 95.5%. With an additional threshold, coming from rank-1-accuracy of closedworld setting, our real-time application reaches as open-world setting a f1-score of 79.44% and a recall of 68.44%.

Lightweight Multimodal Domain Generic Person Reidentification Metric for Person-Following Robots

“…Therefore, in Figure 2 a, to learn a cross-modal reidentification metric, our work takes different original images of N persons and takes their generated images in random poses and in random styles to randomly transform these different training images into different modals, say RGB, Grayscale, and Sketch modals. In our work, RGB images are transformed into Grayscale and Sketch modals due to the reason that in the real world, it is not necessary to always use an RGB sensor on a mobile robot; already, a large number of works have used IR modality [ 12 ]. However, a large number of public reidentification datasets have no IR images; therefore, we opted to transform RGB images into grayscale images.…”

“…Therefore, to overcome these shortcomings in visual trackers [ 11 ], recently, reidentification metrics have been learned and integrated with visual trackers [ 12 ] to follow the target person [ 1 , 2 ]. These reidentification metrics are learned by matching color-histograms and gait features [ 1 , 13 ], as well as extracting deep CNN features to learn deep similarity metrics [ 2 , 3 , 14 , 15 , 16 ].…”

Ablation Study of a Person Re-Identification on a Mobile Robot Using a Depth Camera

“…9. The detection of persons in the depth camera images and cropping them out as described in our previous research [12], stays untouched.…”

“…Once the image processing pipeline is adjusted, we need to optimize the model for the final image input in the same way as it is done for the image Fig. 2c in our previous research [12].…”