The k-ary n-direct s-indirect family of topologies for large-scale interconnection networks

Abstract

In large-scale supercomputers, the interconnection network plays a key role in system performance. Network topology highly defines the performance and cost of the interconnection network. Direct topologies are sometimes used due to its reduced hardware cost, but the number of network dimensions is limited by the physical 3-D space, which leads to an increase of the communication latency and a reduction of network throughput for large machines. Indirect topologies can provide better performance for large machines, but at higher hardware cost. In this paper, we propose a new family of hybrid topologies, the k-ary n-direct s-indirect, that combines the best features from both direct and indirect topologies to efficiently connect an extremely high number of processing nodes. The proposed network is an n-dimensional topology where the k nodes of each dimension are connected through a small indirect topology of s stages. This combination results in a family of topologies that provides high performance, with latency and throughput figures of merit close to indirect topologies, but at a lower hardware cost. In particular, it doubles the throughput obtained per cost unit compared with indirect topologies in most of the cases. Moreover, their fault-tolerance degree is similar to the one achieved by direct topologies built with switches with the same number of ports.

Combining Source-adaptive and Oblivious Routing with Congestion Control in High-performance Interconnects using Hybrid and Direct Topologies

“…In general, the objective of hybrid topologies is to provide high-performance like indirect topologies, but at a similar cost compared to direct topologies. With this objective in mind, the k-ary n-direct s-indirect (KNS) family of topologies were proposed [28]. Specifically, KNS topologies organize end nodes in n dimensions, as in a direct network topology, each dimension having k end nodes, but the end nodes of a given dimension are not interconnected as in meshes or Tori.…”

“…Both properties ease the implementation of efficient routing algorithms, such as the Hybrid-DOR, source-adaptive and oblivious (See Section 2.2). The KNS family of topologies [28] has been evaluated in comparison to other topologies (Tori, Fat-trees, etc. ), comparing cost, performance and complexity, offering a better performance/cost ratio than indirect topologies, if the number of end nodes to connect is high.…”

“…In KNS topologies, a deterministic routing algorithm is proposed derived from DOR, called Hybrid-DOR [28]. The Hybrid-DOR algorithm implements a policy where network dimensions are crossed in an established order to guarantee deadlock-freedom, as it happens with the DOR algorithm in the direct networks.…”

“…These "tailored" queuing schemes are known as topology-and routingaware. In that sense, we proposed the topology-and routing-aware queuing scheme called BBQ (Band-based Queuing) [34], which is tailored to KNS topologies using the Hybrid-DOR deterministic routing algorithm [28]. BBQ significantly reduces the HoL blocking utilizing a small number of queues per port.…”

Head‐of‐line blocking avoidance in Slim Fly networks using deadlock‐free non‐minimal and adaptive routing

“…Among the queuing schemes, some of them, such as VOQnet, 28 VOQsw, 15 DAMQs, 15 DBBM, 14 or DSBM, 29 are topology agnostic, ie, they are designed generally without considering the topology or the routing algorithm. BBQ 18 is tailored to the KNS topology 32 with the Hybrid-DOR routing algorithm. OBQA 16 and vFTree 17 have been devised for fat-tree network topologies using the routing algorithms proposed by Gómez et al 30 and by Zahavi et al, 31 respectively.…”