Understanding Network Saturation Behavior on Large-Scale Blue Gene/P Systems

Balaji, Pavan; Naik, Harish; Desai, Narayan

doi:10.1109/icpads.2009.117

Cited by 6 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[26] examine dynamic routing over content addressable networks and demonstrate that routing can be improved while recording fewer states of the neighbors. A detailed analysis of the congestion behavior on Blue Gene/P interconnection networks is provided by P. Balaji et al [27]. Their experiments execute on over 128K cores providing insight into the network performance characteristics of MPI.…”

Section: Billion-node Torus Network Scaling Studymentioning

confidence: 99%

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

Liu

Carothers

2011

2011 IEEE Workshop on Principles of Advanced and Distributed Simulation

View full text Add to dashboard Cite

Exascale supercomputers will have millions or even hundreds of millions of processing cores and the potential for nearly billionway parallelism. Exascale compute and data storage architectures will be critically dependent on the interconnection network. The most popular interconnection network for current and future supercomputer systems is the torus (e.g., k-ary, n-cube). This paper focuses on the modeling and simulation of ultra-large-scale torus networks using Rensselaer's Optimistic Simulator System (ROSS). We compare real communication delays between our model and the actual torus network from the Blue Gene/L using 2,048 processors. Our performance experiments demonstrate the ability to simulate million to billion-node torus networks. 1 The torus network model for a 16-million-node configuration shows a high degree of strong scaling when going from 1,024 cores to 32,768 cores on Blue Gene/L with a peak event-rate of nearly 5 billion events per second. Finally, we demonstrate the performance of our torus network model configured with 1-billion-nodes using up to 16,384 Blue Gene/L processors.

show abstract

Section: Billion-node Torus Network Scaling Studymentioning

confidence: 99%

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

Liu

Carothers

2011

2011 IEEE Workshop on Principles of Advanced and Distributed Simulation

View full text Add to dashboard Cite

show abstract

“…Other research on communication over mesh and torus topologies which took into account issues of network congestion includes work on MPI collectives [10], topology-aware mapping [11], and network saturation on Blue Gene/P [12]. Our approach to analyzing message aggregation and determining a good message size also shares common elements with work on message strip-mining [13].…”

Section: Related Workmentioning

confidence: 99%

TRAM: Optimizing Fine-Grained Communication with Topological Routing and Aggregation of Messages

Wesolowski

Venkataraman

Gupta

et al. 2014

2014 43rd International Conference on Parallel Processing

View full text Add to dashboard Cite

Abstract-Fine-grained communication in supercomputing applications often limits performance through high communication overhead and poor utilization of network bandwidth. This paper presents Topological Routing and Aggregation Module (TRAM), a library that optimizes fine-grained communication performance by routing and dynamically combining short messages. TRAM collects units of fine-grained communication from the application and combines them into aggregated messages with a common intermediate destination. It routes these messages along a virtual mesh topology mapped onto the physical topology of the network. TRAM improves network bandwidth utilization and reduces communication overhead. It is particularly effective in optimizing patterns with global communication and large message counts, such as all-to-all and many-to-many, as well as sparse, irregular, dynamic or data dependent patterns. We demonstrate how TRAM improves performance through theoretical analysis and experimental verification using benchmarks and scientific applications. We present speedups on petascale systems of 6x for communication benchmarks and up to 4x for applications.

show abstract

“…In previous work [7,8], we noticed the variance of communication performance with process mapping, and we studied the network congestion characteristics of flat torus networks that cause this behavior. In this paper, we extend that research by first performing a detailed analysis of the mapping of logical process layouts on the physical network topology.…”

Section: Introductionmentioning

confidence: 99%

Mapping communication layouts to network hardware characteristics on massive-scale blue gene systems

et al. 2011

Self Cite

View full text Add to dashboard Cite

For parallel applications running on high-end computing systems, which processes of an application get launched on which processing cores is typically determined at application launch time without any information about the application characteristics. As high-end computing systems continue to grow in scale, however, this approach is becoming increasingly infeasible for achieving the best performance. For example, for systems such as IBM Blue Gene and Cray XT that rely on flat 3D torus networks, process communication often involves network sharing, even for highly scalable applications. This causes the overall application performance to depend heavily on how processes are mapped on the network. In this paper, we first analyze the impact of different process mappings on application performance on a massive Blue Gene/P system. Then, we match this analysis with application communication patterns that we allow applications to describe prior to being launched. The underlying process management system can use this combined information in conjunction with the hardware characteristics of the system to determine the best mapping for the application. Our experiments study the performance of different communication patterns, including 2D and 3D nearest-neighbor communication and structured Cartesian grid communication. Our studies, that scale up to 131,072 cores of the largest BG/P system in the United States (using 80% of the total system size), demonstrate that different process mappings can show significant difference in overall performance, especially on scale. For example, we show that this difference can be as much as 30% for P3DFFT and up to twofold for HALO. Through our proposed model, however, such differences in performance can be avoided so that the best possible performance is always achieved.

show abstract

Understanding Network Saturation Behavior on Large-Scale Blue Gene/P Systems

Cited by 6 publications

References 10 publications

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

TRAM: Optimizing Fine-Grained Communication with Topological Routing and Aggregation of Messages

Mapping communication layouts to network hardware characteristics on massive-scale blue gene systems

Contact Info

Product

Resources

About