Tri-modal Person Re-identification with RGB, Depth and Thermal Features

Abstract

Person re-identification is about recognizing people who have passed by a sensor earlier. Previous work is mainly based on RGB data, but in this work we for the first time present a system where we combine RGB, depth, and thermal data for re-identification purposes. First, from each of the three modalities, we obtain some particular features: from RGB data, we model color information from different regions of the body; from depth data, we compute different soft body biometrics; and from thermal data, we extract local structural information. Then, the three information types are combined in a joined classifier. The tri-modal system is evaluated on a new RGB-D-T dataset, showing successful results in re-identification scenarios.

Development of Agent Logic Programming Means for Multichannel Intelligent Video Surveillance

“…The main areas of application of this idea are detection, recognition and reidentification of people in intelligent video surveillance systems [5][6][7][8][9][10][11], medical applications, including remote detection of dangerous diseases by the airport security system [12][13][14][15], support of rescuers [16], interaction people with robots [17], agriculture [18,19], automatic video surveillance for serviceability and safety of equipment operation, thermal imaging odometry [20][21][22]. In this article, as an example, the problem of remote temperature estimation of body parts of people moving in the area of video surveillance will be considered.…”

Person tracking association using multi-modal systems

“…Also, an important research effort has been dedicated to the extraction of features using visual depth sensors [8,10].…”

“…Some examples include RGB-based features such as color, shape, or texture [1,8]; appearance features incorporate histogram, graph model, spatial occurrence model [3]; whereas biometric features comprise face patterns recognition and gait analysis [7,10]. These features allow to describe a person and consequently match them with the one with the most similar detected elements.…”

Person Tracking Using Audio and Depth Cues

“…Combination of other modalities has recently started to become more popular. For instance, [15,22] tracks person from both laser range and camera data; the work in [17] fuses RGB, depth and thermal features. Yet, there are some essential limitations associated with the existing mono-or cross-modal person tracking methods.…”