Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum

Abstract

As an increasing number of Distributed Machine Learning (DML) tasks are hosted on cloud platforms in the edge-cloud continuum, Data Centers (DCs) with massive data and computational requirements have become one of the world’s largest energy consumers, leading to significant carbon emissions. Reducing energy consumption and carbon emissions is an extremely crucial and challenging issue for the sustainable development of cloud service providers. While utilizing renewable energy can help reduce the carbon emissions of DCs, the intermittent and unstable nature still causes DCs to rely heavily on high-carbon brown energy. For the resource-intensive and delay-tolerant DML tasks, this paper introduces multi-renewable energy in the geo-distributed continuum to address this issue, the spatiotemporal complementarity maximizes the renewable energy utilization and compensates for time-dependent energy differences with geographic advantages. Additionally, considering the dynamic differences in carbon intensity and electricity prices across distributed DCs in the continuum, we propose an energy and carbon-aware algorithm called ECMR for scheduling heterogeneous virtual machine creation tasks of DML among multi-clouds in different time zones. It is demonstrated that compared with the baseline methods, the ECMR significantly reduces the total power consumption, energy cost, and carbon emission of data centers while maintaining an acceptable service quality. The utilization of renewable energy in data centers has been significantly improved to 90.8% by flexibly leveraging the spatiotemporal complementarity of multi-renewable energy. Compared with existing competing algorithms, the proposed method exhibits significant improvements with an achieved average response time of 12.6 ms, and a task failure rate of 1.25%.