SimMC: Simple Masked Contrastive Learning of Skeleton Representations for Unsupervised Person Re-Identification

Abstract

The generation of feasible adversarial examples is necessary for properly assessing models that work in constrained feature space. However, it remains a challenging task to enforce constraints into attacks that were designed for computer vision. We propose a unified framework to generate feasible adversarial examples that satisfy given domain constraints. Our framework can handle both linear and non-linear constraints. We instantiate our framework into two algorithms: a gradient-based attack that introduces constraints in the loss function to maximize, and a multi-objective search algorithm that aims for misclassification, perturbation minimization, and constraint satisfaction. We show that our approach is effective in four different domains, with a success rate of up to 100%, where state-of-the-art attacks fail to generate a single feasible example. In addition to adversarial retraining, we propose to introduce engineered non-convex constraints to improve model adversarial robustness. We demonstrate that this new defense is as effective as adversarial retraining. Our framework forms the starting point for research on constrained adversarial attacks and provides relevant baselines and datasets that future research can exploit.

TranSG: Transformer-Based Skeleton Graph Prototype Contrastive Learning with Structure-Trajectory Prompted Reconstruction for Person Re-Identification

“…[6] proposes the AGE model to encode recognizable gait features from 3D skeleton sequences, while its extension SGELA [2] further combines self-supervised semantics learning (e.g., sequence sorting) and sequence contrastive learning to improve discriminative feature learning. The SimMC framework [14] is proposed to encode prototypes and intra-sequence relations of masked skeleton sequences for person re-ID. MG-SCR [18] and SM-SGE [19] perform multi-stage body-component relation learning based on multi-scale graphs to learn person re-ID representations.…”

“…Person re-identification (re-ID) is a challenging task of retrieving and matching a specific person across varying views or scenarios, which has empowered many vital applications such as security authentication, human tracking, and robotics [1][2][3][4]. Recently driven by economical, nonobtrusive and accurate skeleton-tracking devices like Kinect [5], person re-ID via 3D skeletons has attracted surging attention in both academia and industry [2,[6][7][8][9][10][11][12][13][14]. Unlike conventional methods that rely on visual appearance features (e.g., colors, silhouettes), skeleton-based person re-ID methods model unique body and motion representations with 3D positions of key body joints, which typically enjoy smaller data sizes, lighter models, and more robust performance under scale and view variations [2,15].…”

“…Most existing methods manually extract anthropometric descriptors and gait attributes from body-joint coordinates [7][8][9][10]13], or leverage sequence learning paradigms such as Long Short-Term Memory (LSTM) [16] to model body and motion features with skeleton sequences [2,6,14,17]. However, these methods rarely explore the inherent body relations within skeletons (e.g., inter-joint motion correlations), thus largely ignoring some valuable skeleton patterns.…”

“…However, these methods rarely explore the inherent body relations within skeletons (e.g., inter-joint motion correlations), thus largely ignoring some valuable skeleton patterns. To fill this gap, a few recent works [11,18] construct skeleton graphs to model body-component relations in terms of structure and action. These methods typically require multi-stage non-parallel relation modeling, e.g., [18] models collaborative relations conditioned on the structural relations, while they cannot simultaneously mine different underlying relations.…”

“…These methods typically require multi-stage non-parallel relation modeling, e.g., [18] models collaborative relations conditioned on the structural relations, while they cannot simultaneously mine different underlying relations. On the other hand, they usually leverage sequence-level instances such as averaged features of sequential graphs [11] for representation learning, which inherently limits their ability to exploit richer graph semantics from fine-grained (e.g., node-level) representations. For example, there usually exist strong correlations between nearby body-joint nodes within a local spatialtemporal graph context, which can prompt learning unique and recognizable skeleton patterns for person re-ID.…”