Deep action: A mobile action recognition framework using edge offloading

Abstract

Recording users' lives as short-form videos has been an emerging trend with the advance of mobile devices. The videos contain a wealth of information that requires a significant amount of computation to retrieve. In this paper, we propose Deep action, a framework that leverages edge offloading to enable human actions recognition on mobile devices. Deep action first samples frames from a video according to the accuracy requirement. The sampled frames are then compressed and fed into deep learning models to generate an action label. Considering the varying conditions of the wireless connection, we design an online scheduler to strategically offload compressed video snippets to the edge server. Furthermore, we use OpenCL to implement the video compression-related operations on mobile GPU, such that the model inference and video compression can operate in parallel on the mobile device. We implement Deep action on the Android OS and evaluate it on a commercial off-the-shelf mobile device and an edge server. The performance evaluation demonstrates that Deep action brings up to 19 × and 13 × execution speedup, compared to the local-only and remote-only strategies, respectively.

Pipeline valve operation action recognition method based on YOLOv5

“…Although, deep learning rocketed development of HAR, it brought explosive computing growth which meant high performance devices and long-term module training. Luckily, more and more researchers noticed that it is necessary to reduce computation and simplified architecture to make HAR could be deployed on mobile devices or edge devices [15], [16].…”