Computation Scheduling for Distributed Machine Learning With Straggling Workers

Amiri, Mohammad Mohammadi; Gündüz, Deniz

doi:10.1109/tsp.2019.2952051

Cited by 58 publications

(29 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, some other streams of machine scheduling based on machine learning are also being studied. Research on the dynamic pricing algorithm based on reinforcement learning can work effectively without knowing the system dynamics information in advance, and the proposed energy scheduling algorithm can further reduce the system cost [81]. Another path for scheduling based on machine learning is about multi‐agent scheduling, which is currently very scarce.…”

Section: Resultsmentioning

confidence: 99%

Machine learning‐based scheduling: a bibliometric perspective

Cao

et al. 2021

IET Collab Intel Manufact

View full text Add to dashboard Cite

In recent years, the rapid development of artificial intelligence and data science has given rise to the study of data driven algorithms in highly volatile systems. The scheduling of complex shop floor resources falls into such a category, which is often non‐linear in nature, time varying, multi‐objective, and subject to interruptions. Ergo, the machine learning‐based scheduling, has become a research hotspot and attracted the attention of many scholars. In the literature, the research methods employed in solving scheduling problems are based on various perspectives, such as mathematical programming, combinatorial optimization, and heuristic rules. However, due to the inherent complexity of the problem, many issues remain to be addressed. In particular, with the availability of production data, the progress of computing power, and the breakthrough in intelligent algorithms, a novel branch of data driven algorithms present great potential, for example, the deep learning and reinforcement learning‐based algorithms. To reveal the value of machine learning‐based scheduling methods, bibliometric analysis was conducted to analyse the relevant articles and documents from the year 1980 to 2019. Finally, the future research trend in the domain of machine learning‐based scheduling is considered and tips are provided for researchers as well as practitioners to find leading scientists for collaborations.

show abstract

Section: Resultsmentioning

confidence: 99%

Machine learning‐based scheduling: a bibliometric perspective

Cao

et al. 2021

IET Collab Intel Manufact

View full text Add to dashboard Cite

show abstract

“…It happens when large-sized models and datasets with multiple tasks are processed by on-device and battery-limited workers. In this case, as studied in [120], an effective solution could be scheduling the resultant stragglers while offloading their computationally demanding tasks (or even training data with a loss of privacy) to neighbors or edge servers, a conceptual design known as mobile edge computing (MEC) [121], [122]. Such task offloading in MEC needs to take into the account of device heterogeneity [123], communication limitations [124], [125], and demandsupply capabilities of processing power [126] in addition to its impact on the tolerable training latency [122] and target training/inference accuracy [127] while ensuring devices' privacy [128].…”

Section: Scheduling and Offloadingmentioning

confidence: 97%

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

et al. 2021

View full text Add to dashboard Cite

Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. By imbuing intelligence into the network edge, edge nodes can proactively carry out decision-making, and thereby react to local environmental changes and disturbances while experiencing zero communication latency. To achieve this goal, it is essential to cater for high ML inference accuracy at scale under time-varying channel and network dynamics, by continuously exchanging fresh data and ML model updates in a distributed way. Taming this new kind of data traffic boils down to improving the communication efficiency of distributed learning by optimizing communication payload types, transmission techniques, and scheduling, as well as ML architectures, algorithms, and data processing methods. To this end, this article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases. SIGNIFICANCE AND MOTIVATIONThe pursuit of extremely stringent latency and reliability guarantees is essential in the fifth generation (5G) communication system and beyond [1], [2]. In a wirelessly automated factory, the remote control of assembly robots should provision the same level of target latency and reliability offered by existing wired factory systems. To this end, for instance, control packets should be delivered within 1 ms with 99.99999% reliability [3]- [5]. Things are becoming even more challenging in the emerging mission-critical applications beyond 5G. A prime example is the forthcoming nonterrestrial networks consisting of a massive constellation of low-altitude earth orbit (LEO) satellites [6]- [11]. Given such

show abstract

“…In an edge setup with limited resources in communication and computation, this introduces training stragglers degrading the overall training performance. In this view, client scheduling [35]- [37] and computation offloading [38]- [40] with the focus on guaranteeing target training/inference accuracy have been identified as a promising research direction.…”

Section: B Distributed Learning Over Wireless Networkmentioning

confidence: 99%

Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs

Soret

Nguyen

Seeger

et al. 2022

IEEE Trans. on Green Commun. Netw.

View full text Add to dashboard Cite

An Intelligent IoT Environment (iIoTe) is comprised of heterogeneous devices that can collaboratively execute semiautonomous IoT applications, examples of which include highly automated manufacturing cells or autonomously interacting harvesting machines. Energy efficiency is key in such edge environments, since they are often based on an infrastructure that consists of wireless and battery-run devices, e.g., e-tractors, drones, Automated Guided Vehicle (AGV)s and robots. The total energy consumption draws contributions from multiple iIoTe technologies that enable edge computing and communication, distributed learning, as well as distributed ledgers and smart contracts. This paper provides a state-of-the-art overview of these technologies and illustrates their functionality and performance, with special attention to the tradeoff among resources, latency, privacy and energy consumption. Finally, the paper provides a vision for integrating these enabling technologies in energyefficient iIoTe and a roadmap to address the open research challenges.

show abstract

Computation Scheduling for Distributed Machine Learning With Straggling Workers

Cited by 58 publications

References 21 publications

Machine learning‐based scheduling: a bibliometric perspective

Machine learning‐based scheduling: a bibliometric perspective

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs

Contact Info

Product

Resources

About