Benchmarking vision kernels and neural network inference accelerators on embedded platforms

Abstract

Developing efficient embedded vision applications requires exploring various algorithmic optimization trade-offs and a broad spectrum of hardware architecture choices. This makes navigating the solution space and finding the design points with optimal performance trade-offs a challenge for developers. To help provide a fair baseline comparison, we conducted comprehensive benchmarks of accuracy, run-time, and energy efficiency of a wide range of vision kernels and neural networks on multiple embedded platforms: ARM57 CPU, Nvidia Jetson TX2 GPU and Xilinx ZCU102 FPGA. Each platform utilizes their optimized libraries for vision kernels (OpenCV, VisionWorks and xfOpenCV) and neural networks (OpenCV DNN, TensorRT and Xilinx DPU). For vision kernels, our results show that the GPU achieves an energy/ frame reduction ratio of 1.1-3.2 compared to the others for simple kernels. However, for more complicated kernels and complete vision pipelines, the FPGA outperforms the others with energy/frame reduction ratios of 1.2-22.3. For neural networks [Inception-v2 and ResNet-50, ResNet-18, Mobilenet-v2 and SqueezeNet], it shows that the FPGA achieves a speed up of [2.5, 2.1, 2.6, 2.9 and 2.5] and an EDP reduction ratio of [1.5, 1.1, 1.4, 2.4 and 1.7] compared to the GPU FP16 implementations, respectively.

Development of smart camera systems based on artificial intelligence network for social distance detection to fight against COVID-19

“…Therefore, it is necessary to use the model and pre-trained weights together with the most appropriate software to work with the GPU. Studies were conducted on software environments such as TensorFlow, TensorFlow Light, NVIDIA TensorRT, and OpenCV DNN Module [35] . These types of software frameworks allow real-time operation of models with 8-bit integer and 16-bit floating-point optimizations.…”

AMagPoseNet: Real-Time Six-DoF Magnet Pose Estimation by Dual-Domain Few-Shot Learning From Prior Model

“…Table I shows the calculation results of different models, where ResNet-18, a common backbone network in computer vision, is used as a benchmark for comparison. Qasaimeh et al [28] measured the performance of ResNet-18 on embedded platforms and showed that ResNet-18 could achieve 5.17 frames/s on ARM Cortex A57 CPU and 145 frames/s on Jetson TX2 GPU. Compared to ResNet-18, the AMagPoseNet has only 24% of its NPs and 0.98% of its computation (FLOPs).…”

Anatomy of Deep Learning Image Classification and Object Detection on Commercial Edge Devices: A Case Study on Face Mask Detection

“…Reference [20] investigates the on-the-edge inference of DNNs in terms of latency, energy consumption, and temperature, on five different hardware platforms; unlike the proposed method, this work does not take advantage of the optimization frameworks we have investigated. In [21], an in-depth benchmark analysis of three embedded platforms is performed for image vision applications including MobileNet and InceptionV2; in [22], EDLAB is delivered, an end-to-end benchmark to evaluate the overall performance of three image classification and one object detection models across Intel NCS2, Edge TPU and Jetson Xavier NX. In [23], a performance analysis of the edge TPU board is provided for object classification.…”