Host–Parasite: Graph LSTM-in-LSTM for Group Activity Recognition

Shu, Xiangbo; Zhang, Liyan; Sun, Yunlian; Tang, Jinhui

doi:10.1109/tnnls.2020.2978942

Cited by 180 publications

(43 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wu et al [75] proposed global motion pattern to represent complex multi-person motions in the sports video. Global motion patterns extracted by an optical flow algorithm are fed into convolutional neural networks and LSTM networks to extract spatial and temporal features for event classification.…”

Section: Hierarchical Temporal Modelingmentioning

confidence: 99%

A Comprehensive Review of Group Activity Recognition in Videos

Wang

Jian

et al. 2021

Int. J. Autom. Comput.

View full text Add to dashboard Cite

Human group activity recognition (GAR) has attracted significant attention from computer vision researchers due to its wide practical applications in security surveillance, social role understanding and sports video analysis. In this paper, we give a comprehensive overview of the advances in group activity recognition in videos during the past 20 years. First, we provide a summary and comparison of 11 GAR video datasets in this field. Second, we survey the group activity recognition methods, including those based on handcrafted features and those based on deep learning networks. For better understanding of the pros and cons of these methods, we compare various models from the past to the present. Finally, we outline several challenging issues and possible directions for future research. From this comprehensive literature review, readers can obtain an overview of progress in group activity recognition for future studies.

show abstract

Section: Hierarchical Temporal Modelingmentioning

confidence: 99%

A Comprehensive Review of Group Activity Recognition in Videos

Wang

Jian

et al. 2021

Int. J. Autom. Comput.

View full text Add to dashboard Cite

show abstract

“…Hajiaghayi and Vahedi [87] utilized LSTM models prediction and pattern extraction of code failure. In addition, Shu et al [82] proposed a multi-LSTM or LSTM-in-LSTM mechanism for group activity recognition. Denoted as GLIL (Graph LSTM-In-LSTM), they utilized LSTMs for person-level activity recognition, which reside in a global graph LSTM for group-level recognition.…”

Section: Machine Learning On Sequential Datamentioning

confidence: 99%

Towards Efficient and Intelligent Internet of Things Search Engine

et al. 2021

View full text Add to dashboard Cite

The Internet of Things (IoT) has created a novel ecosystem for sensing and actuation throughout our world, enabling intelligently controlled autonomous systems to conserve energy, water crops, manage factories, and provide situation awareness on an unprecedented scale. As IoT progresses, the interest in IoT search engines, that is, search engines to find IoT devices and retrieve IoT data, has grown. While basic examples of IoT search engines exist, considerable challenges prevent the full realization of an efficient and intelligent IoT search engine that provides universal data service, scalable data communication and retrieval, and efficient querying of massively distributed heterogeneous devices and data. In this article, we first propose a generic framework for the IoT search engine, and then present a naming service for the IoT system, an essential component for an effective IoT search engine. We also outline some research challenges and possible solutions for building efficiency and intelligence in the IoT search engine. Further, we present a case study and seek to address a particular aspect of the query process for IoT search, namely efficient and timely query processing. Given the now obvious advances in machine learning, the potential for deep learning-based prediction to improve resource use, and thus query retrieval, is clear. In detail, we utilize Long-Short-Term Memory (LSTM) neural network architecture to predict aggregated query volumes to be preemptively applied and stored for immediate response. Combining several realistic IoT datasets, we explore the efficacy of simultaneously predicting multiple targets for predictive query retrieval.INDEX TERMS Machine learning, Internet of Things, search engine, edge intelligence, applications.

show abstract

“…Shu et al [15] proposed a novel Hierarchical Long Short-Term Concurrent Memory (H-LSTCM) to learn the dynamic inter-related representation among a group of persons for hierarchically recognizing human interactions. Shu et al [16] proposed a novel graph LSTM-in-LSTM (GLIL) framework to address the problem of group activity recognition by modeling the person-level actions and the group-level activity simultaneously. Tang et al [17] proposed a novel Coherence Constrained Graph LSTM (CCG-LSTM) for group activity recognition, by exploring the motion-level characteristics of group activity with several coherence constraints.…”

Section: Related Workmentioning

confidence: 99%

An Encrypted Speech Retrieval Method Based on Deep Perceptual Hashing and CNN-BiLSTM

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

ABSTRACRT Since convolutional neural network (CNN) can only extract local features, and long shortterm memory (LSTM) neural network model has a large number of learning calculations, a long processing time and an obvious degree of information loss as the length of speech increases. Utilizing the characteristics of autonomous feature extraction in deep learning, CNN and bidirectional long short-term memory (BiLSTM) network are combined to present an encrypted speech retrieval method based on deep perceptual hashing and CNN-BiLSTM. Firstly, the proposed method extracts the Log-Mel Spectrogram/MFCC features of the original speech and enters the CNN and BiLSTM networks in turn for model training. Secondly, we use the trained fusion network model to learn the deep perceptual feature and generate deep perceptual hashing sequences. Finally, the normalized Hamming distance algorithm is used for matching retrieval. In order to protect the speech security in the cloud, a speech encryption algorithm based on a 4D hyperchaotic system is proposed. The experimental results show that the proposed method has good discrimination, robustness, recall and precision compared with the existing methods, and it has good retrieval efficiency and retrieval accuracy for longer speech. Meanwhile, the proposed speech encryption algorithm has a high key space to resist exhaustive attacks. INDEX TERMS Encrypted speech retrieval, CNN-BiLSTM, Deep perceptual hashing, Speech feature extraction, 4D hyperchaotic system.

show abstract

Host–Parasite: Graph LSTM-in-LSTM for Group Activity Recognition

Cited by 180 publications

References 42 publications

A Comprehensive Review of Group Activity Recognition in Videos

A Comprehensive Review of Group Activity Recognition in Videos

Towards Efficient and Intelligent Internet of Things Search Engine

An Encrypted Speech Retrieval Method Based on Deep Perceptual Hashing and CNN-BiLSTM

Contact Info

Product

Resources

About