Memory optimization at Edge for Distributed Convolution Neural Network

Naveen, Soumyalatha; Kounte, Manjunath R

doi:10.1002/ett.4648

Cited by 11 publications

(5 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have summarized the state of the art (SOTA) in energy-efficient and DL-based video surveillance studies in Table 1 for the convenience of the reader. These approaches, as highlighted in [18][19][20][21][22], have achieved significant improvements in energy efficiency by leveraging edge computing and optimization mechanisms. This has reduced network bandwidth and response time in IoT-based smart video surveillance systems, enabling effective object detection and analysis of abnormal behavior.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Bazarov Ravshan Ugli,

Mohammed,

et al. 2024

Sensors

View full text Add to dashboard Cite

Video surveillance systems are integral to bolstering safety and security across multiple settings. With the advent of deep learning (DL), a specialization within machine learning (ML), these systems have been significantly augmented to facilitate DL-based video surveillance services with notable precision. Nevertheless, DL-based video surveillance services, which necessitate the tracking of object movement and motion tracking (e.g., to identify unusual object behaviors), can demand a significant portion of computational and memory resources. This includes utilizing GPU computing power for model inference and allocating GPU memory for model loading. To tackle the computational demands inherent in DL-based video surveillance, this study introduces a novel video surveillance management system designed to optimize operational efficiency. At its core, the system is built on a two-tiered edge computing architecture (i.e., client and server through socket transmission). In this architecture, the primary edge (i.e., client side) handles the initial processing tasks, such as object detection, and is connected via a Universal Serial Bus (USB) cable to the Closed-Circuit Television (CCTV) camera, directly at the source of the video feed. This immediate processing reduces the latency of data transfer by detecting objects in real time. Meanwhile, the secondary edge (i.e., server side) plays a vital role by hosting a dynamically controlling threshold module targeted at releasing DL-based models, reducing needless GPU usage. This module is a novel addition that dynamically adjusts the threshold time value required to release DL models. By dynamically optimizing this threshold, the system can effectively manage GPU usage, ensuring resources are allocated efficiently. Moreover, we utilize federated learning (FL) to streamline the training of a Long Short-Term Memory (LSTM) network for predicting imminent object appearances by amalgamating data from diverse camera sources while ensuring data privacy and optimized resource allocation. Furthermore, in contrast to the static threshold values or moving average techniques used in previous approaches for the controlling threshold module, we employ a Deep Q-Network (DQN) methodology to manage threshold values dynamically. This approach efficiently balances the trade-off between GPU memory conservation and the reloading latency of the DL model, which is enabled by incorporating LSTM-derived predictions as inputs to determine the optimal timing for releasing the DL model. The results highlight the potential of our approach to significantly improve the efficiency and effective usage of computational resources in video surveillance systems, opening the door to enhanced security in various domains.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Naveen et al [22] Saving GPU resources Suggested reducing the number of non-contributing parameters.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Bazarov Ravshan Ugli,

Mohammed,

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…For instance, refs. [4][5][6][7][8] made significant achievements in energy efficiency through edge computing and optimizing mechanisms, resulting in a reduced network bandwidth and response time in IoT-based smart video surveillance systems for effective object detection and abnormal behavior analysis. Moreover, ref.…”

Section: Related Workmentioning

confidence: 99%

Cognitive Video Surveillance Management in Hierarchical Edge Computing System with Long Short-Term Memory Model

Ugli

Kim²,

Mohammed

et al. 2023

Sensors

View full text Add to dashboard Cite

Nowadays, deep learning (DL)-based video surveillance services are widely used in smart cities because of their ability to accurately identify and track objects, such as vehicles and pedestrians, in real time. This allows a more efficient traffic management and improved public safety. However, DL-based video surveillance services that require object movement and motion tracking (e.g., for detecting abnormal object behaviors) can consume a substantial amount of computing and memory capacity, such as (i) GPU computing resources for model inference and (ii) GPU memory resources for model loading. This paper presents a novel cognitive video surveillance management with long short-term memory (LSTM) model, denoted as the CogVSM framework. We consider DL-based video surveillance services in a hierarchical edge computing system. The proposed CogVSM forecasts object appearance patterns and smooths out the forecast results needed for an adaptive model release. Here, we aim to reduce standby GPU memory by model release while avoiding unnecessary model reloads for a sudden object appearance. CogVSM hinges on an LSTM-based deep learning architecture explicitly designed for future object appearance pattern prediction by training previous time-series patterns to achieve these objectives. By referring to the result of the LSTM-based prediction, the proposed framework controls the threshold time value in a dynamic manner by using an exponential weighted moving average (EWMA) technique. Comparative evaluations on both simulated and real-world measurement data on the commercial edge devices prove that the LSTM-based model in the CogVSM can achieve a high predictive accuracy, i.e., a root-mean-square error metric of 0.795. In addition, the suggested framework utilizes up to 32.1% less GPU memory than the baseline and 8.9% less than previous work.

show abstract

“…Pruning [11]- [15] removes redundant or unnecessary connections in a neural network. Pruning involves removing individual weights or removing entire neurons or layers.…”

Section: Related Workmentioning

confidence: 99%

“…Naveen et al [14], [15] 2021 For distributing the workload to each IoT edge device, the pre-trained model is pruned to lower the model parameter, and then fused tail partitioning is employed.…”

Section: Papersmentioning

confidence: 99%

Compact optimized deep learning model for edge: a review

Naveen,

Kounte

2023

IJECE

View full text Add to dashboard Cite

<p>Most real-time computer vision applications, such as pedestrian detection, augmented reality, and virtual reality, heavily rely on convolutional neural networks (CNN) for real-time decision support. In addition, edge intelligence is becoming necessary for low-latency real-time applications to process the data at the source device. Therefore, processing massive amounts of data impact memory footprint, prediction time, and energy consumption, essential performance metrics in machine learning based internet of things (IoT) edge clusters. However, deploying deeper, dense, and hefty weighted CNN models on resource-constraint embedded systems and limited edge computing resources, such as memory, and battery constraints, poses significant challenges in developing the compact optimized model. Reducing the energy consumption in edge IoT networks is possible by reducing the computation and data transmission between IoT devices and gateway devices. Hence there is a high demand for making energy-efficient deep learning models for deploying on edge devices. Furthermore, recent studies show that smaller compressed models achieve significant performance compared to larger deep-learning models. This review article focuses on state-of-the-art techniques of edge intelligence, and we propose a new research framework for designing a compact optimized deep learning (DL) model deployment on edge devices.</p>

show abstract

Memory optimization at Edge for Distributed Convolution Neural Network

Cited by 11 publications

References 49 publications

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Cognitive Video Surveillance Management in Hierarchical Edge Computing System with Long Short-Term Memory Model

Compact optimized deep learning model for edge: a review

Contact Info

Product

Resources

About