READ: Reciprocal Attention Discriminator for Image-to-Video Re-identification

Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.

show abstract

“…[131, [133][134][135][136][137][138][139][140] Fig. 3 Channel, spatial, and temporal attention can be regarded as operating on different domains.…”

Section: Introductionmentioning

confidence: 99%

Attention mechanisms in computer vision: A survey

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Multiscale approaches require significantly more operations, and therefore slow down inference, which is contrary to our goal. Attention has also been applied to video-based reid on streams of images [45], [46], [47], [48], [49], [50], [51], [52], [53], [54]. These approaches primarily focus on temporal attention.…”

Section: Related Workmentioning

confidence: 99%

A Little Bit Attention Is All You Need for Person Re-Identification

Eisenbach¹,

Lübberstedt²,

Aganian³

et al. 2023

Preprint

View full text Add to dashboard Cite

Person re-identification plays a key role in applications where a mobile robot needs to track its users over a long period of time, even if they are partially unobserved for some time, in order to follow them or be available on demand. In this context, deep-learning based real-time feature extraction on a mobile robot is often performed on specialpurpose devices whose computational resources are shared for multiple tasks. Therefore, the inference speed has to be taken into account. In contrast, person re-identification is often improved by architectural changes that come at the cost of significantly slowing down inference. Attention blocks are one such example. We will show that some well-performing attention blocks used in the state of the art are subject to inference costs that are far too high to justify their use for mobile robotic applications. As a consequence, we propose an attention block that only slightly affects the inference speed while keeping up with much deeper networks or more complex attention blocks in terms of re-identification accuracy. We perform extensive neural architecture search to derive rules at which locations this attention block should be integrated into the architecture in order to achieve the best trade-off between speed and accuracy. Finally, we confirm that the best performing configuration on a re-identification benchmark also performs well on an indoor robotic dataset.

show abstract

“…As for the I2V person Re-ID, some technologies [8,37] have been studied. Gu et al [7] propose a novel temporal knowledge propagation (TKP) method that first utilizes video representation network to learn the temporal knowledge and then propagates it to the image representation network.…”

Section: Cross-modality Person Re-idmentioning

confidence: 99%

“…However, due to the modality discrepancy, images and videos corresponding to the same identity label may be far away from each other. To this end, existing methods [7,8] mainly develop specific network architectures or training objectives for the feature alignment of images and videos in the shared space. For example, READ [8] proposed an attention-aware discriminator architecture that chooses to aggregate useful spatio-temporal information; TKP [7] devised a temporal knowledge propagation method, which transfers the temporal knowledge of videos to images by a feature-based TKP loss and a distance-based TKP loss.…”

Section: Introductionmentioning

confidence: 99%

Pose‐guided adversarial video prediction for image‐to‐video person re‐identification

He,

Chen,

Pan

2023

IET Image Processing

View full text Add to dashboard Cite

The image‐to‐video (I2V) person re‐identification (Re‐ID) is a cross‐modality pedestrian retrieval task, whose crux is to reduce the large modality discrepancy between images and videos. To this end, this paper proposes to predict the following video frames from a single image. Thus, the I2V person Re‐ID can be transformed to video‐to‐video (V2V) Re‐ID. Considering that predicting video frames from a single image is an ill‐posed problem, this paper proposes two strategies to improve the quality of the predicted videos. First, a pose‐guided video prediction pipeline is proposed. The given single image and pedestrian pose are encoded via image encoder and pose encoder, respectively; then, the image feature and pose feature are concatenated as the input of the video decoder. The authors minimize the difference between the predicted video and true video, and simultaneously minimize the difference between the true pose and predicted pose. Second, the conditional adversarial training strategy is employed to generate high‐quality video frames. Specifically, the discriminator takes the source image as condition and distinguishes whether the input frames are fake or true following frames of the source image. Experimental results demonstrate that the pose‐guided adversarial video prediction can effectively improve accuracy of I2V Re‐ID.

show abstract

READ: Reciprocal Attention Discriminator for Image-to-Video Re-identification

Cited by 12 publications

References 35 publications

Attention mechanisms in computer vision: A survey

Attention mechanisms in computer vision: A survey

A Little Bit Attention Is All You Need for Person Re-Identification

Pose‐guided adversarial video prediction for image‐to‐video person re‐identification

Contact Info

Product

Resources

About