Cynthia

Zheng, Haoyue; Xu, Fei; Chen, Li; Zhou, Zhi; Liu, Fangming

doi:10.1145/3337821.3337873

Cited by 37 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inference (17) Training (10) Both (1) Hardware accelerator (13) Algorithm co-design (8) New numeric formats ( 6) FIGURE 6. Included papers by categories.…”

Section: ) Pruningmentioning

confidence: 99%

“…However, distributed training is less costefficient than training using one machine. Distributed training involves parameter synchronization overhead and can lead to under-utilized cloud VMs waiting for parameter updates from each other [13]. Likewise, the combination of increased inference frequency and near real-time requirement can be resolved by the provision of more cloud VMs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Systematic Literature Review on Cost-Efficient Deep Learning

Klemetti,

Raatikainen,

Myllyaho

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Cloud computing and deep learning, the recent big trends in the software industry, have enabled small companies to scale their business up rapidly. However, this growth is not without a costdeep learning models are related to the heaviest workloads in cloud data centers. When the business grows, the monetary cost of deep learning in the cloud grows fast as well. Deep learning practitioners should be prepared and equipped to limit the growing cost. We performed a systematic literature review on the methods to control the monetary cost of deep learning. Our library search resulted in 16066 papers from three article databases, IEEE Xplore, ACM Digital Library, and Scopus. We narrowed them down to 112 papers that we categorized and summarized. We found that: 1) Optimizing inference has raised more interest than optimizing training. Popular deep learning libraries already support some of the inference optimization methods such as quantization, pruning, and teacher-student. 2) The research has been centered around image inputs, and there seems to be a research gap for other types of inputs. 3) The research has been hardwareoriented, and the most typical approach to control the cost of deep learning is based on algorithm-hardware co-design. 4) Offloading some of the processing to client devices is gaining interest and has the potential to reduce the monetary cost of deep learning.

show abstract

“…Inference (17) Training (10) Both (1) Hardware accelerator (13) Algorithm co-design (8) New numeric formats ( 6) FIGURE 6. Included papers by categories.…”

Section: ) Pruningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Systematic Literature Review on Cost-Efficient Deep Learning

Klemetti,

Raatikainen,

Myllyaho

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Both DNN training and inference are computation intensive. Efficient DNN training has been extensively studied [33], [34], [35], [36], [37], [38] and we focus on DNN inference in this paper. DNN-based mobile applications typically impose strict SLA requirements such as constrained end-toend latency [8], [22], [39].…”

Section: Deep Learningmentioning

confidence: 99%

HiTDL: High-Throughput Deep Learning Inference at the Hybrid Mobile Edge

Wang

Pei

et al. 2022

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

“…[25] shows how to do cluster resource allocation and scheduling for ML training jobs by developing and using an empirical performance model to determine number of workers and parameter servers to use. Similarly, Cynthia [26] uses an analytical performance model for cost efficient cloud resource provisioning. In contrast, our approach can directly start training without the need for apriori modeling.…”

Section: Resnetmentioning

confidence: 99%

Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching

Tyagi

Sharma

2020

2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)

View full text Add to dashboard Cite

Current techniques and systems for distributed model training mostly assume that clusters are comprised of homogeneous servers with a constant resource availability. However, cluster heterogeneity is pervasive in computing infrastructure, and is a fundamental characteristic of low-cost transient resources (such as EC2 spot instances). In this paper, we develop a dynamic batching technique for distributed data-parallel training that adjusts the mini-batch sizes on each worker based on its resource availability and throughput. Our mini-batch controller seeks to equalize iteration times on all workers, and facilitates training on clusters comprised of servers with different amounts of CPU and GPU resources. This variable mini-batch technique uses proportional control and ideas from PID controllers to find stable mini-batch sizes. Our empirical evaluation shows that dynamic batching can reduce model training times by more than 4× on heterogeneous clusters.

show abstract

Cynthia

Cited by 37 publications

References 20 publications

Systematic Literature Review on Cost-Efficient Deep Learning

Systematic Literature Review on Cost-Efficient Deep Learning

HiTDL: High-Throughput Deep Learning Inference at the Hybrid Mobile Edge

Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching

Contact Info

Product

Resources

About