Cognitive Video Surveillance Management in Hierarchical Edge Computing System with Long Short-Term Memory Model

Abstract

Nowadays, deep learning (DL)-based video surveillance services are widely used in smart cities because of their ability to accurately identify and track objects, such as vehicles and pedestrians, in real time. This allows a more efficient traffic management and improved public safety. However, DL-based video surveillance services that require object movement and motion tracking (e.g., for detecting abnormal object behaviors) can consume a substantial amount of computing and memory capacity, such as (i) GPU computing resources for model inference and (ii) GPU memory resources for model loading. This paper presents a novel cognitive video surveillance management with long short-term memory (LSTM) model, denoted as the CogVSM framework. We consider DL-based video surveillance services in a hierarchical edge computing system. The proposed CogVSM forecasts object appearance patterns and smooths out the forecast results needed for an adaptive model release. Here, we aim to reduce standby GPU memory by model release while avoiding unnecessary model reloads for a sudden object appearance. CogVSM hinges on an LSTM-based deep learning architecture explicitly designed for future object appearance pattern prediction by training previous time-series patterns to achieve these objectives. By referring to the result of the LSTM-based prediction, the proposed framework controls the threshold time value in a dynamic manner by using an exponential weighted moving average (EWMA) technique. Comparative evaluations on both simulated and real-world measurement data on the commercial edge devices prove that the LSTM-based model in the CogVSM can achieve a high predictive accuracy, i.e., a root-mean-square error metric of 0.795. In addition, the suggested framework utilizes up to 32.1% less GPU memory than the baseline and 8.9% less than previous work.

Pedestrian Abnormal Behavior Detection System Using Edge–Server Architecture for Large–Scale CCTV Environments

“…This provides a new solution to the multi-layer response regression problem and suggests adding a new level of accuracy and efficiency to the reliability assessment of complex systems. In [1], we propose a method to leverage LSTM models to predict object appearance patterns in a hierarchical edge computing environment for analyzing surveillance video in systems that analyze surveillance video, such as CCTV. The proposed method is proper when object occurrences are infrequent because it saves GPU memory by loading the model only when needed and releasing the model when no objects are detected.…”

“…The system focuses on detecting and predicting potential collisions between pedestrians and vehicles in a real-time edge vision environment. In reference [1], a long short-term memory (LSTM) [8] model on an edge computing system is used to recognize and predict the appearance patterns of objects in surveillance videos. In particular, it applies optimization to improve the system's efficiency and real-time processing power.…”

“…This approach is intended to make the system scalable and easily adaptable to different environments and requirements. A similar approach is adopted in existing research [1,2]. In [1], the authors describe a method that analyzes the edge device and then integrates the results into the cloud.…”

“…A similar approach is adopted in existing research [1,2]. In [1], the authors describe a method that analyzes the edge device and then integrates the results into the cloud. However, these methods can be less accurate because they rely on analytics results from the edge and do not utilize high-performance analytics models.…”

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

“…This constant switching between model states can significantly increase the delay and impose other operational issues. Another framework named CogVSM was proposed in [12], incorporating predictive modeling and smoothing techniques to control the threshold value (i.e., θ m ) for releasing the DL model. According to the claim in the CogVSM framework, the Long Short-Term Memory (LSTM) model predicts future object occurrences based on historical data, and these predictions are then passed to smooth the LSTM predictions using the Exponential Weighted Moving Average (EWMA) technique.…”