Hierarchical Relational Networks for Group Activity Recognition and Retrieval

Fusing motion patterns and key visual information for semantic event recognition in basketball videos

“…More recently, deep neural network architecture achieves substantial success on group activity understanding due to its high-capacity of multi-level feature representation and integration [1,2,3,4,13,14]. Wang et al [2] proposed a recurrent interactional context model to aggregate person level, group level and scene level interactions.…”

“…Deng et al [13] performed structure learning by a unified framework of integrating graphical models and a sequential inference network. Sport Video Analysis: Recently, a considerable amount of efforts have been devoted to team sports analysis, such as basketball [1], volleyball [3,4,5,14,15,16,50,51,52], soccer [19,20], water polo [18], ice hockey [21] etc. Ramanathan et al [1] introduced an attention based BLSTM network to identify the most relevant component (key player) of the corresponding event and recognize basketball events.…”

“…Ramanathan et al [1] introduced an attention based BLSTM network to identify the most relevant component (key player) of the corresponding event and recognize basketball events. Ibrahim et al [3] built a relationship graph guided network to infer relational representations of each player and encode scene level representations. Wu et al [46] proposed a domain knowledge based basketball semantic events classification method which represented global and collective motion patterns on different event stages for event recognition.…”

“…The main challenge is how to extract discriminative and robust spatio-temporal contextual features in the dynamic scenes. A great deal of attempts utilized various modalities of data to establish the mapping relationship between the group activity and semantic representation such as key components [1], multi-level interaction representations [2], hierarchical relational representations [3] and semantic graph [4].…”

Group Activity Recognition by Using Effective Multiple Modality Relation Representation With Temporal-Spatial Attention

“…In recent years, collective activity recognition has made great progress with the development of deep learning [2], [9], [19], [20], [25], [29], [32], [38]. Typically, they first extract person-level features using a convolutional neural network (CNN).…”