Forecasting GPU Type Allocation via Demand Feature Extraction in Heterogeneous Clusters

Abstract

The availability of multiple types of GPUs enhances parallel processing capabilities and provides users with a broader spectrum of options in heterogeneous GPU clusters. However, the inherent randomness and uncertainty in GPU type requests introduce complexity into resource allocation, resulting in disparities between requested and allocated GPU types and exacerbating imbalances in the GPU resource distribution. In this study, we perform a comprehensive analysis of the attributes associated with diverse GPU requests and allocations in heterogeneous clusters. By introducing the adaptive selection model (ASM), our approach not only predicts the demand for various GPU types but also extracts critical features from the resource demand. These features play a pivotal role in predicting GPU resource allocation, effectively bridging the gap between request and allocation. Moreover, we introduce spatial and temporal long short-term memory (ST-LSTM), a novel LSTM-based model that accommodates both spatial and temporal correlations in GPU type allocation. Our experimental efforts confirm the effectiveness of our proposed methodology through the utilization of real-world trace data from Alibaba cloud data centres. The ASM achieves an impressive accuracy rate of 87% and illuminates the key factors influencing user GPU type selection. Additionally, the ST-LSTM model demonstrates excellent performance with average RMSE and MAE values of 1.84 and 1.09, respectively.