Uncertainty Modeling with Second-Order Transformer for Group Re-identification

Abstract

Group re-identification (G-ReID) focuses on associating the group images containing the same persons under different cameras. The key challenge of G-ReID is that all the cases of the intra-group member and layout variations are hard to exhaust. To this end, we propose a novel uncertainty modeling, which treats each image as a distribution depending on the current member and layout, then digs out potential group features by random samplings. Based on potential and original group features, uncertainty modeling can learn better decision boundaries, which is implemented by two modules, member variation module (MVM) and layout variation module (LVM). Furthermore, we propose a novel second-order transformer framework (SOT), which is inspired by the fact that the position modeling in the transformer is coped with the G-ReID task. SOT is composed of the intra-member module and inter-member module. Specifically, the intra-member module extracts the first-order token for each member, and then the inter-member module learns a second-order token as a group feature by the above first-order tokens, which can be regarded as the token of tokens. A large number of experiments have been conducted on three available datasets, including CSG, DukeGroup and RoadGroup. The experimental results show that the proposed SOT outperforms all previous state-of-the-art methods.

Unsupervised Group Re-identification via Adaptive Clustering-Driven Progressive Learning

“…However, due to the increase or decrease of members and the changes of the relative positions in the group, the appearance features of groups may change dramatically, limiting the effectiveness of these methods. SOT (Zhang et al 2022b) and 3DT (Zhang et al 2022a) respectively propose new modeling methods that better solve the problems of group layout and membership changes. However, these fully supervised methods require fine-grained labeling of both the groups and members.…”

“…The RoadGroup dataset contains 324 monitor images, including 162 group classes. We follow the division of these two datasets for training and testing as described in (Zhang et al 2022b). The i-LIDS MCTS dataset contains 274 monitor images, including 64 group classes.…”

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

“…Predicting pedestrians' future behavior is approached through two distinct modeling methods, which differ in terms of their output. The first approach involves the prediction or classification of the future action of a pedestrian, i.e., whether he will cross the street in the near future, often referred to as intention prediction [7,[12][13][14][15][16][17][18][19][20][21][22][23][24][25]. In the second approach, the focus lies on predicting the future trajectory of a pedestrian for a defined prediction horizon [2][3][4][5][6][7]10,14,[26][27][28][29][30][31][32][33][34][35].…”

“…Other approaches model spatio-temporal features by first extracting features using CNNs [10,12,15] or graph structures [6,13,20] and processing them later on with RNNs. Recently, partially attention-based [2,15,23,24] transformers [7,8,25,35,38], as well as goaldriven [4,5,9,33] approaches, have also gained growing interest.…”

“…Related work may also be differentiated by the information used as input to predict the future behavior of pedestrians. The pedestrian's motion history is the most fundamental information as almost all methods rely on the past motion of the pedestrian independent of the domain [2][3][4][5][6][7][8][9][10][12][13][14][15][16][22][23][24][25][26][27][28][29][30][31][32][33][34]36,37], This information includes bounding boxes for ego-centric camera view methods or global coordinates for methods in bird's eye view. Additionally, features like the distance to the ego-vehicle [26,27], semantic and contextual information [7,13,14,20,23,24,26,27,31,32,36], as well as visual or appearance features [2,3,6,10,…”