Hiding communication latency and coherence overhead in software DSMs

Abstract

In this paper we propose the use of a PCI-based programmable protocol controller for hiding communication and coherence overheads in software DSMs. Our protocol controller provides three different types of overhead tolerance: a) moving basic communication and coherence tasks away from computation processors; b) prefetching of diffs; and c) generating and applying diffs with hardware assistance. We evaluate the isolated and combined impact of these features on the performance of TreadMarks. We also compare performance against two versions of the Shrimp-based AURC protocol. Using detailed execution-driven simulations of a 16-node network of workstations, we show that the greatest performance benefits provided by our protocol controller come from our hardware-supported diffs. Reducing the burden of communication and coherence transactions on the computation processor is also beneficial but to a smaller extent. Prefetching is not always profitable. Our results show that our protocol controller can improve running time performance by up to 50% for TreadMarks, which means that it can double the TreadMarks speedups. The overlapping implementation of TreadMarks performs as well or better than AURC for 5 of our 6 applications. We conclude that the simple hardware support we propose allows for the implementation of high-performance software DSMs at low cost. Based on this conclusion, we are building the NCP 2 parallel system at COPPE/UFRJ.

Addressing a workload characterization study to the design of consistency protocols

“…A important amount of research focusing on this topic [5,11,16,[19][20][21][22] have addressed reducing the impact of asynchronous communication in SVM systems. The next section discusses asynchronous communication, summarizing key results from recent research in this area.…”

“…For example, the client node asynchronously communicates with the server node to read a given page or to lock a given semaphore. This communication involves a context switch in the server node, which implies high service latencies and wastes precious computing time in the server [19].…”

“…Some of these mechanisms include hardware support that partially, or totally, avoids this kind of communication [5,[19][20][21]. Others try to reduce this communication or hide its latency using software techniques [11,16,22].…”

“…Hardware techniques can update data utilizing the same techniques as have been implemented in software, as well as perform data requests automatically without processor intervention [5,19]. In [19], hardware support for an LRC-based protocol that services data requests was proposed.…”

“…In [19], hardware support for an LRC-based protocol that services data requests was proposed. The hardware changes needed to support this protocol in hardware is small since the processor is already interrupted to perform metadata maintenance tasks of related data structures.…”