Temporal Reasoning Graph for Activity Recognition

Abstract

Despite great success has been achieved in activity analysis, it still has many challenges. Most existing work in activity recognition pay more attention to design efficient architecture or video sampling strategy. However, due to the property of fine-grained action and long term structure in video, activity recognition is expected to reason temporal relation between video sequences. In this paper, we propose an efficient temporal reasoning graph (TRG) to simultaneously capture the appearance features and temporal relation between video sequences at multiple time scales. Specifically, we construct learnable temporal relation graphs to explore temporal relation on the multi-scale range. Additionally, to facilitate multi-scale temporal relation extraction, we design a multihead temporal adjacent matrix to represent multi-kinds of temporal relations. Eventually, a multi-head temporal relation aggregator is proposed to extract the semantic meaning of those features convolving through the graphs. Extensive experiments are performed on widely-used large-scale datasets, such as Something-Something and Charades, and the results show that our model can achieve state-of-the-art performance. Further analysis shows that temporal relation reasoning with our TRG can extract discriminative features for activity recognition.

Analysis of Volleyball Video Intelligent Description Technology Based on Computer Memory Network and Attention Mechanism

“…It refers to the technology of object recognition of images to identify targets and objects of different modes [ 25 ]. If the action in volleyball video needs to be extracted by intelligent analysis and description, it needs to be realized by intelligent algorithm in image recognition technology [ 26 ]. With the development of artificial intelligence technology, in terms of data processing of video information, image recognition technology based on deep learning algorithm has become a research hotspot and development trend in the fields of video and image information mining and data association analysis [ 27 , 28 ].…”

Interaction-Aware Spatio-Temporal Pyramid Attention Networks for Action Classification

“…However, the slowfast network is computationally expensive to deal with the dynamics and the temporal scale of actions at the input frame level [65,66]. Zhang et al [67] propose an efficient temporal reasoning graph for action recognition by simultaneously capturing the appearance features and temporal relations between video sequences at multiple time scales. The temporal reasoning graph extracts discriminative features for action recognition.…”

SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network

“…Object detection serves as a core problem in computer vision, which has shown dramatic progress recently. Further, object detection methods have been implemented into various related tasks as a pre-processing step, including image captioning [48], [49], scene graph [50], [51], visual reasoning [52], [53], person re-identification [54], [55], etc. Object detection methods can be roughly divided into one-stage methods (e.g., YOLO [56], SSD [57]) and two-stage methods (e.g., R-CNN [58], Fast R-CNN [59], Faster R-CNN [60]) according to whether region proposals are generated.…”