A Design Space Exploration Framework for Convolutional Neural Networks Implemented on Edge Devices

Abstract

Deploying convolutional neural networks (CNNs) in embedded devices that operate at the edges of Internet of Things (IoT) networks provides various advantages in terms of performance, energy efficiency, and security in comparison with the alternative approach of transmitting large volumes of data for processing to the cloud. However, the implementation of CNNs on low power embedded devices is challenging due to the limited computational resources they provide and to the large resource requirements of state-of-the-art CNNs. In this paper, we propose a framework for the efficient deployment of CNNs in low power processor-based architectures used as edge devices in IoT networks. The framework leverages design space exploration (DSE) techniques to identify efficient implementations in terms of execution time and energy consumption. The exploration parameter is the utilization of hardware resources of the edge devices. The proposed framework is evaluated using a set of 6 state-of-the-art CNNs deployed in the Intel/Movidius Myriad2 low power embedded platform. The results show that using the maximum available amount of resources is not always the optimal solution in terms of performance and energy efficiency. Fine-tuned resource management based on DSE, reduces the execution time up to 3.6% and the energy consumption up to 7.7% in comparison with straightforward implementations.

A Hybrid Performance Prediction Approach for Fully-Connected Artificial Neural Networks on Multi-core Platforms

“…The approach in [3] focuses on performing Neural Architecture Search on embedded GPU platforms under accuracy, timing and energy constraints. Another notable approach is [18], which proposes a latency and energy evaluation flow to find optimized mappings of neural networks on edge devices. These approaches focus on the evaluation of the neural network through implementation and testing on real prototype.…”

Investigating the Interaction between Energy Consumption, Quality of Service, Reliability, Security, and Maintainability of Computer Systems and Networks

“…Throughout the SDK4ED project, several empirical studies have been conducted for reaching conclusions with respect to potential trade-offs among the three quality attributes of choice [22,[30][31][32]34]. Based on the identified trade-offs, a multi-criteria decision-making (MCDM) model that leverages concepts from fuzzy logic has been developed [38], providing information about the impacts of the suggested refactorings. Similarly to the other toolboxes, the MCDM model is implemented in the form of a web service.…”

BenchPress

“…Predictive modeling for compiler optimisation heuristics has been shown to outperform human experts and reduce development time in previous studies [5,7,34,36]. Predictive models learn such heuristics by training on source-level benchmarks or on static code features extracted at the (1) syntax level by traversing their Astract Syntax Tree (AST) or (2) Intermediate Representation (IR) with the help of compiler passes, as shown in Figure 1.…”