Scheduling for broadcast operation in heterogeneous parallel computing environments

Abstract

SUMMARYParallel computing utilizes heterogeneous parallel computing environments in which multiple processors of various kinds are connected by communication links. Efficient collective communication operations are needed to realize efficient parallel computing in parallel computing environments. Although methods for realizing efficient collective communication operations have previously been proposed for homogeneous parallel computing environments, these methods applied as they are to heterogeneous parallel computing environments cannot realize efficient collective communication operations. In this paper, we consider design of an efficient broadcast, which is one of the most important collective communication operations. We show that we can extend the known sufficient condition for Fastest-Node-First (FNF) to output an optimal schedule, where FNF is an efficient heuristic algorithm for the broadcast. We show that we can reduce the time cost of the branch-and-bound search for the optimal schedule.

Broadcast scheduling for wide-area parallel distributed systems

“…For a heterogeneous parallel distributed system whose network topology is represented by a complete graph, FNF (first-node first) outputs a highly efficient broadcast schedule [9][10][11]. Consider first the case in which FNF is applied to a system whose network topology is represented by a split graph in which the order of the independent vertexes is 1.…”

“…Studies of broadcast scheduling in recent years have focused on heterogeneous parallel distributed systems in which the time required for data communications is nonuniform [7][8][9][10][11]. Consequently, this study also considers broadcast scheduling by restricting the topology of the heterogeneous parallel distributed system to a split graph in which the order of the independent vertex is 1.…”

“…Most previous studies of this problem have discussed the relation between the network topology and the complexity of the problem [3][4][5][6][7], or have considered a system with complete graph topology from the viewpoint of application to real systems [8][9][10][11]. Not many studies have considered wide-area parallel distributed systems in which multiple clusters are connected.…”

“…There can be various patterns of broadcast scheduling, but scheduling that can be completed in a short time is desirable in a high-performance system. The problem of finding the schedule with the shortest completion time is called the minimum broadcast time problem (MBT), or the broadcast scheduling problem, and has been the subject of many studies [2][3][4][5][6][7][8][9][10][11].…”

Broadcast Trees for Heterogeneous Platforms

“…They introduce the Fastest Node First (FNF) heuristic: to construct a good broadcast tree, it is better to put fastest processors (processors that have the smallest sending time) at the top of tree. Some theoretical results (NP-completeness and approximation algorithms) have been developed for the problem of broadcasting a message in this model: see [15,21,25,26,31]. The (bidirectional) one-port model; which we extensively used in this paper, has been introduced by Bhat et al [8,7].…”