DeepMon

Huynh, Loc N.; Lee, Mi Kyung; Balan, Rajesh Krishna

doi:10.1145/3081333.3081360

Cited by 259 publications

(15 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to using SIMD units, DeepX [62] and uLayer [63] proposed to decompose deep model network architectures into unitblocks of various types and deploy them on heterogeneous local device processors (e.g., CPUs, GPUs). DeepMon [64] proposed to use various optimization techniques including the convolutional layer caching, decomposition, and matrix multiplication optimizations to efficiently offload convolutional layers to mobile GPUs and accelerate the processing. Unfortunately, hardware accelerators like GPUs are not common and energy-efficient, and the software approaches mentioned above can not reduce the number of operations, so the performance improvement is limited.…”

Section: Related Workmentioning

confidence: 99%

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

Gong

Wang

et al. 2023

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) have been widely deployed, while traditional cloud data-centers based applications suffer from the bandwidth and latency network demand when applying to Industrial-Internet-of-Things (IIoT) fields. It is critical to migrate the CNNs inference to edge devices for efficiency and security concerns. However, it is challenging to deploy complex CNNs on resource-constraint IIoT edge devices due to a large number of parameters and intensive floating-point computations. In this paper, we propose ABM-SpConv-SIMD, an on-device inference optimization framework, aiming at accelerating the network inference by fully utilizing the low-cost and common CPU resource. ABM-SpConv-SIMD first adopts a model optimizer with pruning and quantization, which produces Sparse Convolutional models. And then, the Accumulation-Before-Multiplication mechanism is proposed to reduce multiplication operations. Additionally, the SIMD instructions, which are commonly available on cost-effective edge devices, are employed to improve the performance of convolutions. We have implemented ABM-SpConv-SIMD base on the ARM Compute Library software framework and evaluated on Hikey970 and Raspberry Pi devices with two representative models AlexNet and ResNet50. The results show that the ABM-SpConv-SIMD can significantly improve the performance, and achieve on average of 1.96x and 1.73x speedup respectively over the baseline implementation with negligible loss of accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

Gong

Wang

et al. 2023

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…To mitigate/avoid these problems, [6], [7] suggested ondevice resource management. DeepMon [6] aims to guarantee continuous vision apps by optimizing the convolutional neural networks (CNN) on mobile GPUs. It accelerated the convolution by reusing the intermediate results via caching.…”

Section: A Resource Management For Multiple Vision Appsmentioning

confidence: 99%

DynaMIX: Resource Optimization for DNN-Based Real-Time Applications on a Multi-Tasking System

Cho¹,

Shin²

2023

Preprint

View full text Add to dashboard Cite

As deep neural networks (DNNs) prove their importance and feasibility, more and more DNN-based apps, such as detection and classification of objects, have been developed and deployed on autonomous vehicles (AVs). To meet their growing expectations and requirements, AVs should "optimize" use of their limited onboard computing resources for multiple concurrent in-vehicle apps while satisfying their timing requirements (especially for safety). That is, real-time AV apps should share the limited on-board resources with other concurrent apps without missing their deadlines dictated by the frame rate of a camera that generates and provides input images to the apps. However, most, if not all, of existing DNN solutions focus on enhancing the concurrency of their specific hardware without dynamically optimizing/modifying the DNN apps' resource requirements, subject to the number of running apps, owing to their high computational cost. To mitigate this limitation, we propose DynaMIX (Dynamic MIXed-precision model construction), which optimizes the resource requirement of concurrent apps and aims to maximize execution accuracy. To realize a real-time resource optimization, we formulate an optimization problem using app performance profiles to consider both the accuracy and worstcase latency of each app. We also propose dynamic model reconfiguration by lazy loading only the selected layers at runtime to reduce the overhead of loading the entire model. DynaMIX is evaluated in terms of constraint satisfaction and inference accuracy for a multi-tasking system and compared against stateof-the-art solutions, demonstrating its effectiveness and feasibility under various environmental/operating conditions.

show abstract

“…Sun et al presented a scarified backpropagation technique for neural network learning based on GPU to achieve acceleration [34]. Loc et al proposed a mobile deep learning framework named DeepMon with GPU computing to execute the DNN for continuous video [35]. Song et al presented a distributed and dynamically tuned framework with GPU computing for CNN based big data processing and achieve acceleration [36].…”

Section: A Gpu Computingmentioning

confidence: 99%

Efficient ResNet Model to Predict Protein-Protein Interactions With GPU Computing

Hong

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Protein-protein interactions (PPI) play an important role in the cell activities of organisms. The deep research about PPI can help humans understand the mechanism of life activities and apply protein functions better. Nowadays, PPI prediction algorithms based on amino acid sequences using the recurrent neural network (RNN) can overcome the disadvantages of traditional biological experimental methods and achieve high accuracy. However, these algorithms are usually time-consuming and cannot take full advantage of graphics processing units (GPU) with efficient computation performance to accelerate PPI prediction, because the RNN model considers the time series of sequences. In this paper, we propose an efficient algorithm based on the residual network (ResNet) model to predict PPI (ResPPI). Our algorithm uses the embedding method to represent amino acid sequences, combining the advantages of powerful feature extraction capabilities of the ResNet with deep layers and GPU performance. The experimental results show that the ResPPI algorithm can ensure high accuracy and reduce training time greatly. Based on the ordinary GPU device, compared with the state-of-the-art LSTM model, the speed of the ResPPI algorithm is five times faster than that of the LSTM, whereas the ResPPI algorithm can achieve similar accuracy to the LSTM. Besides, in the case of unbalanced datasets, the ResPPI algorithm can perform better. INDEX TERMS Protein-protein interactions (PPI), residual network (ResNet), graphics processing units (GPU) computing, training speed.

show abstract

DeepMon

Cited by 259 publications

References 31 publications

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

DynaMIX: Resource Optimization for DNN-Based Real-Time Applications on a Multi-Tasking System

Efficient ResNet Model to Predict Protein-Protein Interactions With GPU Computing

Contact Info

Product

Resources

About