Identifying the Culprits Behind Network Congestion

Abstract

Abstract-Network congestion is one of the primary causes of performance degradation, performance variability and poor scaling in communication-heavy parallel applications. However, the causes and mechanisms of network congestion on modern interconnection networks are not well understood. We need new approaches to analyze, model and predict this critical behavior in order to improve the performance of large-scale parallel applications. This paper applies supervised learning algorithms, such as forests of extremely randomized trees and gradient boosted regression trees, to perform regression analysis on communication data and application execution time. Using data derived from multiple executions, we create models to predict the execution time of communication-heavy parallel applications. This analysis also identifies the features and associated hardware components that have the most impact on network congestion and in turn, on execution time. The ideas presented in this paper have wide applicability: predicting the execution time on a different number of cores, or different input datasets, or even for an unknown code, identifying the best configuration parameters for an application, and finding the root causes of network congestion on different architectures. I. MOTIVATION AND IMPACTNetwork congestion is widely recognized as one of the primary causes of performance degradation, performance variability, and poor scaling in communication-heavy applications running on supercomputers [5]. However, due to the complex nature of interconnection networks, as well as message injection and routing strategies, network congestion and its root causes for network resources and hardware components are not well understood. This makes the problem of mitigating and avoiding network congestion difficult. It also complicates the task of writing congestionavoiding and congestion-minimizing algorithms for communication and task mapping. Therefore, we need new approaches to understand and model network congestion in order to improve the performance of large-scale parallel applications.When a message is sent from one node to another, it is split into packets that pass through many resources and hardware components on the network. A packet starts in an injection FIFO on the source. It then passes through multiple network links and receive buffers on intermediate nodes before it finally lands in the reception FIFO on the destination. When shared by multiple packets, any or all of these network components can slow down individual flits, packets and messages. This paper aims to identify the hardware components that affect the performance of sending a message the most.Our approach is based on using supervised machine learning to build models that map from independent variables, representing different network hardware components, to a dependent variable -the execution time of the application. We only consider computationally balanced, communicationheavy parallel applications and, hence, focus on the communication fraction of the total executi...

Unraveling Network-Induced Memory Contention: Deeper Insights with Machine Learning

“…Of particular relevance to our work, in [4], Bhatele et. al used machine learning to identify sources of network congestion and their success inspired us to employ machine learning techniques in this work, though we are exploring different phenomena.…”

“…We used 100 estimators 4 (trees in the forest), with separate runs for each feature set. That is, we ran the regression for each feature set in isolation.…”

“…Random forests have a measure of feature importance that is calculated differently 4. Trials of 300 trees did not show a significant increase to OOB scores.…”

Machine Learning-Based Modeling for Performance Improvement in an Exascale Systems

“…It is mostly getting more common to adopt corresponding tools (scikit-learn, caffe, TensorFlow, and so on) from machine learning (ML) for intelligence analysis and pattern invention. This has enabled researchers to build predictive models, conclude complex correlation patterns, determine fields of interest for performance optimization, and detect irregular patterns (Bhowmick, S., Eijkhout, V., Freund, Y., Fuentes, E., & Keyes, D., 2006;Sukhija, N., Malone, B., Srivastava, S., Banicescu, I., & Ciorba, F. M., 2014;Bhatele, A., Titus, A. R., et al, 2015, May;Yeom, J. S., Thiagarajan, J. J., et al, 2016;Islam, T. Z., Thiagarajan, J. J., et al, 2016, November).…”

Leveraging Cloud Heterogeneity for Cost-Efficient Execution of Parallel Applications

“…Public cloud providers such as Amazon's EC2 and Microsoft's Azure provide a large number of cloud instance types with different numbers of cores, processing speeds, and network interconnections [18]. Research in this area focuses mostly on porting applications to the cloud [12], evaluating their performance and cost efficiency [13,18], and improving communication performance [4,3,2].…”