A Dynamic Scaling Scheme of Cloud-based DNN Training Clusters

Oh, Seungmin; Kim, Kyeonglok; Seo, Euiseong

doi:10.1109/smartcloud49737.2020.00039

2020 IEEE International Conference on Smart Cloud (SmartCloud) 2020

DOI: 10.1109/smartcloud49737.2020.00039

|View full text |Cite

A Dynamic Scaling Scheme of Cloud-based DNN Training Clusters

Seungmin Oh

Kyeonglok Kim

Euiseong Seo

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Reliable and efficient RAR-based distributed model training in computing power network

Chen,

Li,

Natalino

et al. 2024

J. Opt. Commun. Netw.

View full text Add to dashboard Cite

The computing power network (CPN) is a novel network technology that integrates computing power from the cloud, edge, and terminals using IP/optical cross-layer networks for distributed computing. CPNs can provide an effective solution for distributed model training (DMT). As a bandwidth optimization architecture based on data parallelism, ring all-reduce (RAR) is widely used in DMT. However, any node or link failure on the ring can interrupt or block the requests deployed on the ring. Meanwhile, due to the resource competition of batch RAR-based DMT requests, inappropriate scheduling strategies will also lead to low training efficiency or congestion. As far as we know, there is currently no research that considers the survivability of rings in scheduling strategies for RAR-based DMT. To fill this gap, we propose a scheduling scheme for RAR-based DMT requests in CPNs to optimize the allocation of computing and wavelength resources considering the time dimension while ensuring reliability. In practical scenarios, service providers may focus on different performance metrics. We formulate an integer linear programming (ILP) model and a RAR-based DMT deployment algorithm (RDDA) to solve this problem considering four optimization objectives under the premise of the minimum blocking rate: minimum computing resource consumption, minimum wavelength resource consumption, minimum training time, and maximum reliability. Simulation results demonstrate that our model satisfies the reliability requirements while achieving corresponding optimal performance for DMT requests under four optimization objectives.

show abstract

Reliable and efficient RAR-based distributed model training in computing power network

Chen,

Li,

Natalino

et al. 2024

J. Opt. Commun. Netw.

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

A Dynamic Scaling Scheme of Cloud-based DNN Training Clusters

Cited by 1 publication

References 6 publications

Reliable and efficient RAR-based distributed model training in computing power network

Reliable and efficient RAR-based distributed model training in computing power network

Contact Info

Product

Resources

About