DNN Model Deployment on Distributed Edges

Cho, Eunho; Yoon, Juyeon; Baek, Daehyeon; Lee, Dongman; Bae, Doo‐Hwan

doi:10.1007/978-3-030-92231-3_2

Cited by 3 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These do not account for communication demands on the edge. Others abstract model layers into certain "execution units, " [10,29] which they then choose to slice based on certain resource requirements. Li et al [28] regressively predict a layer's latency demand and optimizes communication bandwidth accordingly.…”

Section: Edge Inferencementioning

confidence: 99%

See 1 more Smart Citation

Partitioning and Placement of Deep Neural Networks on Distributed Edge Devices to Maximize Inference Throughput

Parthasarathy¹,

Krishnamachari

2022

2022 32nd International Telecommunication Networks and Applications Conference (ITNAC)

View full text Add to dashboard Cite

Edge inference has become more widespread, as its diverse applications range from retail to wearable technology. Clusters of networked resource-constrained edge devices are becoming common, yet no system exists to split a DNN across these clusters while maximizing the inference throughput of the system. Additionally, no production-ready orchestration system exists for deploying said models over such edge networks which adopts the robustness and scalability of the cloud. We present an algorithm which partitions DNNs and distributes them across a set of edge devices with the goal of minimizing the bottleneck latency and therefore maximizing inference throughput. The system scales well to systems of different node memory capacities and numbers of nodes, while being node fault-tolerant. We find that we can reduce the bottleneck latency by 10x over a random algorithm and 35% over a greedy joint partitioning-placement algorithm, although the joint-partitioning algorithm outperforms our algorithm in most practical use-cases.Furthermore we find empirically that for the set of representative models we tested, the algorithm produces results within 9.2% of the optimal bottleneck latency. We then developed a standalone cluster network emulator on which we tested configurations of up to 20 nodes and found a steady increase in throughput and decrease in end-to-end latency as the cluster size scales. In these tests, we observed that our system has multi-node fault-tolerance as well as network and system IO fault-tolerance. We have implemented our framework in open-source software that is publicly available to the research community at https://github.com/ANRGUSC/SEIFER.

show abstract

Section: Edge Inferencementioning

confidence: 99%

“…For each model, we ran Algorithm 3 with a certain number of nodes, number of bandwidth classes, and node memory capacity. We used the set of nodes [5,10,15,20,50]. We used the set of bandwidth classes [2,5,8,11,14,17,20].…”

Section: Algorithm Simulationsmentioning

confidence: 99%

Partitioning and Placement of Deep Neural Networks on Distributed Edge Devices to Maximize Inference Throughput

Parthasarathy¹,

Krishnamachari

2022

2022 32nd International Telecommunication Networks and Applications Conference (ITNAC)

View full text Add to dashboard Cite

show abstract

Towards Resource-Efficient DNN Deployment for Traffic Object Recognition: From Edge to Fog

Stojanovic,

Sentic,

Stojanovic

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The paper focuses on the challenges associated with deploying deep neural networks (DNNs) for the recognition of traffic objects using the camera of Android smartphones. The main objective of this research is to achieve resource-awareness, enabling efficient utilization of computational resources while maintaining high recognition accuracy. To achieve this, a methodology is proposed that leverages the Edge-to-Fog paradigm to distribute the inference workload across multiple tiers of the distributed system architecture. The evaluation was conducted using a dataset comprising real-world traffic scenarios and diverse traffic objects. The main findings of this research highlight the feasibility of deploying DNNs for traffic object recognition on resource-constrained Android smartphones. The proposed Edge-to-Fog methodology demonstrated improvements in terms of both recognition accuracy and resource utilization, and viability of both edge-only and edge-fog based approaches. Moreover, the experimental results showcased the adaptability of the system to dynamic traffic scenarios, thus ensuring real-time recognition performance even in challenging environments.

show abstract

ChainNet: A Customized Graph Neural Network Model for Loss-Aware Edge AI Service Deployment

Niu,

Roveri,

Casale

2024

2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)

View full text Add to dashboard Cite

DNN Model Deployment on Distributed Edges

Cited by 3 publications

References 5 publications

Partitioning and Placement of Deep Neural Networks on Distributed Edge Devices to Maximize Inference Throughput

Partitioning and Placement of Deep Neural Networks on Distributed Edge Devices to Maximize Inference Throughput

Towards Resource-Efficient DNN Deployment for Traffic Object Recognition: From Edge to Fog

ChainNet: A Customized Graph Neural Network Model for Loss-Aware Edge AI Service Deployment

Contact Info

Product

Resources

About