QoS in InfiniBand subnetworks

Abstract

Abstract-The InfiniBand Architecture (IBA) has been proposed as an industry standard both for communication between processing nodes and I/O devices and for interprocessor communication. It replaces the traditional bus-based interconnect with a switch-based network for connecting processing nodes and I/O devices. It is being developed by the InfiniBand SM Trade Association (IBTA) in the aim to provide the levels of reliability, availability, performance, scalability, and quality of service (QoS) required by present and future server systems. For this purpose, IBA provides a series of mechanisms that are able to guarantee QoS to the applications. In previous papers, we have proposed a strategy to compute the InfiniBand arbitration tables. In one of these, we presented and evaluated our proposal to treat traffic with bandwidth requirements. In another, we evaluated our strategy to compute the InfiniBand arbitration tables for traffic with delay requirements, which is a more complex task. In this paper, we will evaluate both these proposals together. Furthermore, we will also adapt these proposals in order to treat VBR traffic without QoS guarantees, but achieving very good results. Performance results show that, with a correct treatment of each traffic class in the arbitration of the output port, all traffic classes reach their QoS requirements.

Comparing the latency performance of the DTable and DRR schedulers

“…In [2], we explained how to configure table-based schedulers (in that case for IBA) to provide bandwidth and latency guarantees. In order to provide a flow with a minimum bandwidth, the number of table entries assigned to that flow must accomplish with the proportion of desired egress link bandwidth.…”

“…These table-based schedulers can provide a good latency performance with a low computational complexity. However, these schedulers do not work properly with variable packet sizes and face the problem of bounding the bandwidth and latency assignments [9,2]. In [8] we reviewed these problems and proposed a new tablebased scheduler, that works properly with variable packet sizes.…”

Providing quality of service over advanced switching

“…Note that the bandwidth unused by the control and QoS SCs would be redistributed by the MinBW scheduler among the best-effort SCs. Table scheduler In [3], we explained how to configure this kind of arbitration table (in that case for InfiniBand) to provide bandwidth and latency guarantees. In order to provide traffic of a given VC with a minimum bandwidth, the number of table entries assigned to that VC must accomplish with the proportion of desired egress link bandwidth.…”

“…By fixing this separation, it is possible to control the maximum latency of a network element crossing, and therefore, given a maximum number of hops, the maximum end-to-end latency. In order to choose the maximum separation, the maximum time must be studied that a packet can spend crossing a network element as well as the time it takes to be transmitted to the next element once it has been chosen by the scheduler [3]. Note that the control SC does not have maximum priority when using this scheduler, so we will consider it as any other SC with high latency requirements.…”

Providing Full QoS Support in Clusters Using Only Two VCs at the Switches

“…These are traffic segregation with service levels, the use of VCs (IBA ports can have up to 16 VCs) and the arbitration at output ports according to an arbitration table. Although IBA does not specify how these mechanisms should be used, some proposals have been made to provide applications with QoS in InfiniBand networks [9].…”