A Scalable Parallel Debugging Library with Pluggable Communication Protocols

Abstract

Parallel debugging faces challenges in both scalability and efficiency. A number of advanced methods have been invented to improve the efficiency of parallel debugging. As the scale of system increases, these methods highly rely on a scalable communication protocol in order to be utilized in large-scale distributed environments. This paper describes a debugging middleware that provides fundamental debugging functions supporting multiple communication protocols. Its pluggable architecture allows users to select proper communication protocols as plug-ins for debugging on different platforms. It aims to be utilized by various advanced debugging technologies across different computing platforms. The performance of this debugging middleware is examined on a Cray XE Supercomputer with 21,760 CPU cores.MRNet and SCI provide different tree topologies, communication features and launch methods. We compare them as follows. A. MRNet (Multicast/Reduction Network)MRNet is a software overlay network using a tree of communication processes to connect FE and BE nodes. Its communication tree can be utilized to broadcast/multicast messages downstream and collect or aggregate messages upstream. The tree organization is configurable and it supports common network layouts like k-ary and k-nomial trees, or custom layouts tailored to the specific requirements.Its communication is achieved by using the filter, stream, and communicator components. A communicator represents a group of BE nodes. A stream is a logical channel that connects the FE with the BE nodes of a communicator. Each stream may be attached with a filter that can modify data transferred across it. Message aggregation can be realized via programming filters. Both synchronous and asynchronous ways of receiving messages are provided in MRNet.MRNet supports an attachment mode to create a communication tree. Specifically, MRNet creates only internal processes, while BE processes are created by either a system management or job scheduling tools. The BE processes, after being created by an external service, attaches to the tree that is instantiated by MRNet.

Relative debugging for a highly parallel hybrid computer system

“…Merging ranges takes advantage of the tree's topology. We utilize MRNet's back-end attachment mode [8] to initialize a communication tree, which guarantees the MPI ranks are ordered sequentially across the leaf nodes. Therefore, the MPI rank / thread id space owned by any sub-tree on the same level has no intersection, as shown in Figure 5.…”

Intralayer Communication for Tree-Based Overlay Networks

“…GTI provides a pluggable communication system that allows the use of different communication protocols. An extension of this approach could allow GTI to utilize existing TBON communication systems as in SPDL [17].…”

Continuous Debugging of Microservices

“…The closest near matches are works on debugging the programs running on supercomputers [9], the BigDebugger for debugging MapReduce applications for Big Data analysis [11], and two more recent developments: Cloud Debugger for Google's Cloud Platform for microservices [14] and Squash [17]. They are discussed below.…”

“…To enable the debugging of parallel programs running on supercomputer architectures, Jin et al [9] developed a code library for launching a debug facility simultaneously on the nodes in a supercomputer from the front-end, and sending data collected back to the developer's workstation. The main function of the library code is the communication between the front-end and back-end in supercomputer systems.…”