EosDNN: An Efficient Offloading Scheme for DNN Inference Acceleration in Local-Edge-Cloud Collaborative Environments

Min, Xue; Wu, Huaming; Li, Ruidong; Xu, Minxian; Jiao, Pengfei

doi:10.1109/tgcn.2021.3111731

Cited by 36 publications

(10 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Obviously, the DNNs with DAG structures like GoogleNet and ResNet are more complex. Xue et al [15,16] preprocess the topological DNN model reasonably by designing a two-step strategy for offloading large-scale DNN models in a localedge-cloud collaborative environment. In [17], Wu et al [19] to maximize the user requests throughput.…”

Section: Related Workmentioning

confidence: 99%

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Tian

Hui

et al. 2023

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Deep Neural Network (DNN) has become an essential technology for edge intelligence. Due to significant resource and energy requirements for large-scale DNNs’ inference, executing them directly on energy-constrained Internet of Things (IoT) devices is impractical. DNN partitioning provides a feasible solution for this problem by offloading some DNN layers to execute on the edge server. However, the resources of edge servers are also typically limited. An energy-constrained and resource-constrained optimization problem is generated in such a realistic environment. Motivated by this, we investigate an optimization problem of DNN partitioning and offloading in a multiuser resource-constrained environment, which is considered an intractable Mixed-Integer Nonlinear Problem (MINLP). We decompose the problem into two subproblems and propose an Energy-Efficient DNN Partitioning and Offloading (EEDPO) strategy to solve it in polynomial time based on the minimum cut/maximum flow theorem and dynamic programming. Finally, we test the impact of energy constraint, DNN type, and device number on the performance of EEDPO. Simulation results on realistic DNN models demonstrate that the proposed strategy can significantly improve the DNN inference task completion rate compared to other methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Tian

Hui

et al. 2023

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Under the premise of satisfying the quality of service, the efficiency of inference is improved by coordinating the calculation of heterogeneous equipment. Xue et al [61] propose a DNN inference accelerated offloading scheme in cloud-edge-device collaborative environment. It comprehensively considers large-scale model partition plan and migration plan, reduces inference latency and optimizes DNN real-time query performance.…”

Section: Total Inference Latency Minimizationmentioning

confidence: 99%

A Survey on Collaborative DNN Inference for Edge Intelligence

Ren¹,

Qu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the vigorous development of artificial intelligence (AI), the intelligent applications based on deep neural network (DNN) change people's lifestyles and the production efficiency. However, the huge amount of computation and data generated from the network edge becomes the major bottleneck, and traditional cloud-based computing mode has been unable to meet the requirements of real-time processing tasks. To solve the above problems, by embedding AI model training and inference capabilities into the network edge, edge intelligence (EI) becomes a cutting-edge direction in the field of AI. Furthermore, collaborative DNN inference among the cloud, edge, and end device provides a promising way to boost the EI. Nevertheless, at present, EI oriented collaborative DNN inference is still in its early stage, lacking a systematic classification and discussion of existing research efforts. Thus motivated, we have made a comprehensive investigation on the recent studies about EI oriented collaborative DNN inference. In this paper, we firstly review the background and motivation of EI. Then, we classify four typical collaborative DNN inference paradigms for EI, and analyze the characteristics and key technologies of them. Finally, we summarize the current challenges of collaborative DNN inference, discuss the future development trend and provide the future research direction.

show abstract

“…Among them, edge offloading is a distributed computing paradigm that provides computing services for edge caching, edge training, and edge inference. By integrating methods such as Distributed Machine Learning (DML), Deep Reinforcement Learning (DRL) and Collaborative Machine Learning (CML) into the edge computing, it is beneficial to cope with the explosive growth of communication and computing of emerging IoT applications [187], and achieve the energy-efficient and real-time processing [188].…”

Section: Intelligent Edgementioning

confidence: 99%

AI for Next Generation Computing: Emerging Trends and Future Directions

Gill,

Xu,

Ottaviani

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomic computing investigates how systems can achieve (user) specified "control" outcomes on their own, without the intervention of a human operator. Autonomic computing fundamentals have been substantially influenced by those of control theory for closed and open-loop systems. In practice, complex systems may exhibit a number of concurrent and inter-dependent control loops. Despite research into autonomic models for managing computer resources, ranging from individual resources (e.g., web servers) to a resource ensemble (e.g., multiple resources within a data center), research into integrating Artificial Intelligence (AI) and Machine Learning (ML) to improve resource autonomy and performance at scale continues to be a fundamental challenge. The integration of AI/ML to achieve such autonomic and self-management of systems can be achieved at different levels of granularity, from full to human-in-the-loop automation. In this article, leading academics, researchers, practitioners, engineers, and scientists in the fields of cloud computing, AI/ML, and quantum computing join to discuss current research and potential future directions for these fields. Further, we discuss challenges and opportunities for leveraging AI and ML in next generation computing for emerging computing paradigms, including cloud, fog, edge, serverless and quantum computing environments.

show abstract

EosDNN: An Efficient Offloading Scheme for DNN Inference Acceleration in Local-Edge-Cloud Collaborative Environments

Cited by 36 publications

References 43 publications

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

A Survey on Collaborative DNN Inference for Edge Intelligence

AI for Next Generation Computing: Emerging Trends and Future Directions

Contact Info

Product

Resources

About