Seq2Seq RNN based Gait Anomaly Detection from Smartphone Acquired Multimodal Motion Data

Bonetto, Riccardo; Soldan, Mattia; Lanaro, Alberto; Milani, Simone; Rossi, Michele

doi:10.48550/arxiv.1911.08608

Cited by 4 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before the advent of Transformers, the "Seq2Seq model" [36][37][38][39] based on RNNs was recognized as the dominant methodology for the processing of sequence data.…”

Section: Transformer Mechanismmentioning

confidence: 99%

Three-Stage Deep Learning Framework for Video Surveillance

Lee,

Kang

2024

Applied Sciences

View full text Add to dashboard Cite

The escalating use of security cameras has resulted in a surge in images requiring analysis, a task hindered by the inefficiency and error-prone nature of manual monitoring. In response, this study delves into the domain of anomaly detection in CCTV security footage, addressing challenges previously encountered in analyzing videos with complex or dynamic backgrounds and long sequences. We introduce a three-stage deep learning architecture designed to detect abnormalities in security camera videos. The first stage employs a pre-trained convolutional neural network to extract features from individual video frames. Subsequently, these features are transformed into time series data in the second stage, utilizing a blend of bidirectional long short-term memory and multi-head attention to analyze short-term frame relationships. The final stage leverages relative positional embeddings and a custom Transformer encoder to interpret long-range frame relationships and identify anomalies. Tested on various open datasets, particularly those with complex backgrounds and extended sequences, our method demonstrates enhanced accuracy and efficiency in video analysis. This approach not only improves current security camera analysis but also shows potential for diverse application settings, signifying a significant advancement in the evolution of security camera monitoring and analysis technologies.

show abstract

“…Before the advent of Transformers, the "Seq2Seq model" [36][37][38][39] based on RNNs was recognized as the dominant methodology for the processing of sequence data.…”

Section: Transformer Mechanismmentioning

confidence: 99%

Three-Stage Deep Learning Framework for Video Surveillance

Lee,

Kang

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…These approaches leverage the power of deep neural networks to automatically learn representations and infer correlations between time series [25][26][27][28]. Recurrent neural networks (RNNs) [29][30][31], such as the long short-term memory (LSTM) network [32,33], have shown promise in capturing long-term dependencies and temporal correlations in time series data. These can effectively model sequential information and have been successfully applied to anomaly detection tasks.…”

Section: Introductionmentioning

confidence: 99%

Generative Adversarial Network-Based Anomaly Detection and Forecasting with Unlabeled Data for 5G Vertical Applications

Zhang,

Chen,

Zhang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

With the development of 5G vertical applications, a huge amount of unlabeled network data can be collected, which can be employed for evaluating the user experience and network operation status, such as the identifications and predictions of network anomalies. However, it is challenging to achieve highly accurate evaluation results using the conventional statistical methods due to the limitations of data quality. In this paper, generative adversarial network (GAN)-based anomaly detection and forecasting are studied for 5G vertical applications, which can provide considerable detection and prediction results with unlabeled network data samples. First, the paradigm and deployment of the deep-learning-based anomaly detection and forecasting scheme are designed. Second, the network structure and the training strategy are introduced to fully explore the potential of the GAN model. Finally, the experimental results of our proposed GAN model are provided based on the practical unlabeled network operation data in various 5G vertical scenarios, which show that our proposed scheme can achieve significant performance gains for network anomaly detection and forecasting.

show abstract

“…Sensor fusion plays a pivotal role in many applications. In the human gait system, senor fusion methods are deployed to study gait dynamics [17][18][19][20][21], detect gait anomalies [22][23][24] and control prosthetics [25][26][27][28][29][30]. Numerous sensors are uti-Email addresses: sbalakrishnan@ucsb.edu (Shara Balakrishnan), aqib@ucsb.edu (Aqib Hasnain), rob.egbert@pnnl.gov (Rob Egbert), eyeung@ucsb.edu (Enoch Yeung).…”

Section: Introductionmentioning

confidence: 99%

The Effect of Sensor Fusion on Data-Driven Learning of Koopman Operators

Balakrishnan,

Hasnain,

Egbert

et al. 2021

Preprint

View full text Add to dashboard Cite

Dictionary methods for system identification typically rely on one set of measurements to learn governing dynamics of a system. In this paper, we investigate how fusion of output measurements with state measurements affects the dictionary selection process in Koopman operator learning problems. While prior methods use dynamical conjugacy to show a direct link between Koopman eigenfunctions in two distinct data spaces (measurement channels), we explore the specific case where output measurements are nonlinear, non-invertible functions of the system state. This setup reflects the measurement constraints of many classes of physical systems, e.g., biological measurement data, where one type of measurement does not directly transform to another. We propose output constrained Koopman operators (OC-KOs) as a new framework to fuse two measurement sets. We show that OC-KOs are effective for sensor fusion by proving that when learning a Koopman operator, output measurement functions serve to constrain the space of potential Koopman observables and their eigenfunctions. Further, low-dimensional output measurements can be embedded to inform selection of Koopman dictionary functions for high-dimensional models. We propose two algorithms to identify OC-KO representations directly from data: a direct optimization method that uses state and output data simultaneously and a sequential optimization method. We prove a theorem to show that the solution spaces of the two optimization problems are equivalent. We illustrate these findings with a theoretical example and two numerical simulations.

show abstract

Seq2Seq RNN based Gait Anomaly Detection from Smartphone Acquired Multimodal Motion Data

Cited by 4 publications

References 36 publications

Three-Stage Deep Learning Framework for Video Surveillance

Three-Stage Deep Learning Framework for Video Surveillance

Generative Adversarial Network-Based Anomaly Detection and Forecasting with Unlabeled Data for 5G Vertical Applications

The Effect of Sensor Fusion on Data-Driven Learning of Koopman Operators

Contact Info

Product

Resources

About