Multi-tenant mobile offloading systems for real-time computer vision applications

Fang, Zhou; Lin, Jeng-Hau; Srivastava, Mani; Gupta, Rajesh K.

doi:10.1145/3288599.3288634

Cited by 15 publications

(3 citation statements)

References 83 publications

(154 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The unconstrained Lagrangian function was adopted as a reward function to cope with various constraints. The DQN-based scheduler was also successfully used in FL systems and mobile edge computing systems [117]- [119] to schedule participating devices. For example, Zhou et al [117] adopted DQN to schedule training batches to optimize the quality of query response for a cloud-enabled DNN inference system.…”

Section: Application 1: Stochastic Integer Programming Problemsmentioning

confidence: 99%

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

The sixth generation (6G) wireless systems are envisioned to enable the paradigm shift from "connected things" to "connected intelligence", featured by ultra high density, largescale, dynamic heterogeneity, diversified functional requirements and machine learning capabilities, which leads to a growing need for highly efficient intelligent algorithms. The classic optimization-based algorithms usually require highly precise mathematical model of data links and suffer from poor performance with high computational cost in realistic 6G applications. Based on domain knowledge (e.g., optimization models and theoretical tools), machine learning (ML) stands out as a promising and viable methodology for many complex large-scale optimization problems in 6G, due to its superior performance, generalizability, computational efficiency and robustness. In this paper, we systematically review the most representative "learning to optimize" techniques in diverse domains of 6G wireless networks by identifying the inherent feature of the underlying optimization problem and investigating the specifically designed ML frameworks from the perspective of optimization. In particular, we will cover algorithm unrolling, learning to branch-andbound, graph neural network for structured optimization, deep reinforcement learning for stochastic optimization, as well as endto-end learning for semantic optimization, for solving challenging large-scale optimization problems arising from various important wireless applications. To enable ML implementation in distributed wireless networks across massive number of end devices, federated learning for distributed optimization will further be presented. Through the in-depth discussion, we shed light on the excellent performance of ML-based optimization algorithms with respect to the classical methods, and provide insightful guidance to develop advanced ML techniques in 6G networks. Neural network design, theoretical tools of different ML methods,

show abstract

Section: Application 1: Stochastic Integer Programming Problemsmentioning

confidence: 99%

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…First, we only consider onloading between two devices and assume models already reside in both ends. Model distribution [29] and client multi-tenancy [11] on the server side are interesting adjacent issues, but we do not tackle them in this study.…”

Section: Limitations and Future Workmentioning

confidence: 99%

DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device

Almeida

Laskaridis

Venieris

et al. 2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Recently, there has been an explosive growth of mobile and embedded applications using convolutional neural networks (CNNs). To alleviate their excessive computational demands, developers have traditionally resorted to cloud offloading, inducing high infrastructure costs and a strong dependence on networking conditions. On the other end, the emergence of powerful SoCs is gradually enabling on-device execution. Nonetheless, low- and mid-tier platforms still struggle to run state-of-the-art CNNs sufficiently. In this paper, we present DynO, a distributed inference framework that combines the best of both worlds to address several challenges, such as device heterogeneity, varying bandwidth and multi-objective requirements. Key components that enable this are its novel CNN-specific data packing method, which exploits the variability of precision needs in different parts of the CNN when onloading computation, and its novel scheduler that jointly tunes the partition point and transferred data precision at run time to adapt inference to its execution environment. Quantitative evaluation shows that DynO outperforms the current state-of-the-art, improving throughput by over an order of magnitude over device-only execution and up to 7.9 × over competing CNN offloading systems, with up to 60 × less data transferred.

show abstract

“…First, we only consider onloading between two devices and assume models already reside in both ends. Model distribution [20] and multi-tenancy [7] are interesting adjacent issues, but we do not tackle them in this study.…”

Section: Limitations and Future Workmentioning

confidence: 99%

DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device

Almeida,

Laskaridis,

Venieris

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, there has been an explosive growth of mobile and embedded applications using convolutional neural networks (CNNs). To alleviate their excessive computational demands, developers have traditionally resorted to cloud offloading, inducing high infrastructure costs and a strong dependence on networking conditions. On the other end, the emergence of powerful SoCs is gradually enabling on-device execution. Nonetheless, low-and mid-tier platforms still struggle to run state-of-the-art CNNs sufficiently. In this paper, we present DynO, a distributed inference framework that combines the best of both worlds to address several challenges, such as device heterogeneity, varying bandwidth and multi-objective requirements. Key components that enable this are its novel CNN-specific data packing method, which exploits the variability of precision needs in different parts of the CNN when onloading computation, and its novel scheduler that jointly tunes the partition point and transferred data precision at run time to adapt inference to its execution environment. Quantitative evaluation shows that DynO outperforms the current state-of-the-art, improving throughput by over an order of magnitude over device-only execution and up to 7.9× over competing CNN offloading systems, with up to 60× less data transferred.

show abstract

Multi-tenant mobile offloading systems for real-time computer vision applications

Abstract: Bradshaw at the Cloud Dataflow team for their patient and helpful mentorship and guidance in my internship projects. Nobody has been more important to me than my family. I would like to thank my parents and grandparents, whose love and support are always with me anytime and anywhere.

Cited by 15 publications

References 83 publications

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device

DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device

Contact Info

Product

Resources

About