EDLAB: A Benchmark for Edge Deep Learning Accelerators

Abstract

A new trend tends to deploy deep learning algorithms to edge environments to mitigate privacy and latency issues from cloud computing. Diverse edge deep learning accelerators are devised to speed up the inference of deep learning algorithms on edge devices. Various edge deep learning accelerators feature different characteristics in terms of power and performance, which make it a very challenging task to efficiently and uniformly compare different accelerators. In this paper, we introduce EDLAB, an end-to-end benchmark, to evaluate the overall performance of edge deep learning accelerators. EDLAB consists of state-of-the-art deep learning models, a unified workload preprocessing and deployment framework, as well as a collection of comprehensive metrics.In addition, we propose parameterized models to model the hardware performance bound so that EDLAB can identify the hardware potentials and the hardware utilization of different deep learning applications. Finally, we employ EDLAB to benchmark three edge deep learning accelerators and analyze the benchmarking results. From the analysis we obtain some insightful observations that can guide the design of efficient deep learning applications.

EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems

“…The importance factors in the white dashed box correspond to the weight that needs to be "removed". Subsequently, an automatic script [12] is employed to measure the real latency of the compressed model on the target edge device. It is worth noting that the latency test and model training are conducted separately on the cloud and edge device, respectively, and are executed in parallel without any mutual interference.…”

Anatomy of Deep Learning Image Classification and Object Detection on Commercial Edge Devices: A Case Study on Face Mask Detection

“…Reference [20] investigates the on-the-edge inference of DNNs in terms of latency, energy consumption, and temperature, on five different hardware platforms; unlike the proposed method, this work does not take advantage of the optimization frameworks we have investigated. In [21], an in-depth benchmark analysis of three embedded platforms is performed for image vision applications including MobileNet and InceptionV2; in [22], EDLAB is delivered, an end-to-end benchmark to evaluate the overall performance of three image classification and one object detection models across Intel NCS2, Edge TPU and Jetson Xavier NX. In [23], a performance analysis of the edge TPU board is provided for object classification.…”

Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning

“…However, modern CNNs are usually equipped with billions of operations. For example, the most popular CNN model, ResNet50 [1], has 4.1 billion Multiply-Accumulate operations (MACs), which are computationally prohibitive for embedded hardware [4].…”