Benchmarking of DL Libraries and Models on Mobile Devices

Abstract

Deploying deep learning (DL) on mobile devices has been a notable trend in recent years. To support fast inference of on-device DL, DL libraries play a critical role as algorithms and hardware do. Unfortunately, no prior work ever dives deep into the ecosystem of modern DL libs and provides quantitative results on their performance. In this paper, we first build a comprehensive benchmark that includes 6 representative DL libs and 15 diversified DL models. We then perform extensive experiments on 10 mobile devices, which help reveal a complete landscape of the current mobile DL libs ecosystem. For example, we find that the best-performing DL lib is severely fragmented across different models and hardware, and the gap between those DL libs can be rather huge. In fact, the impacts of DL libs can overwhelm the optimizations from algorithms or hardware, e.g., model quantization and GPU/DSP-based heterogeneous computing. Finally, atop the observations, we summarize practical implications to different roles in the DL lib ecosystem. CCS Concepts• General and reference → Measurement; • Human-centered computing → Ubiquitous and mobile devices.

Automation Slicing and Testing for in-App Deep Learning Models

“…Then the work [45] investigates how iApp providers protect in-App DL models. The work [8] and work [54] perform comprehensive studies on DL model inference performance. The work [24] studies the robustness of the in-App DL models.…”

“…Therefore, it is difficult for us to locate the entry point and exit point of the IO processing as the slicing criterions. In contrast, the DL framework's invoking interfaces, which are used to load DL models and perform universal DL computation, have significant static characteristics [54]. ASTM utilizes these invoking interfaces to prepare slicing criterions for the forward and backward slicing, respectively.…”