The architectural costs of streaming I/O: A comparison of workstations, clusters, and SMPs

Abstract

We investigate resource usage while performing streaming I/O by contrasting three architectures, a single workstation, a cluster, and an SMP, under various I/O benchmarks. We derive analytical and empiricallybased models of resource usage during data transfer, examining the I/O bus, memory bus, network, and processor of each system. By investigating each resource in detail, we assess what comprises a wellbalanced system for these workloads.We find that the architectures we study are not well balanced for streaming I/O applications. Across the platforms, the main limitation to attaining peak performance is the CPU, due to lack of data locality. Increasing processor performance (especially with improved block operation performance) will be of great aid for these workloads in the future. For a cluster workstation, the I/O bus is a major system bottleneck, because of the increased load placed on it from network communication. A well-balanced cluster workstation should have copious I/O bus bandwidth, perhaps via multiple I/O busses. The SMP suffers from poor memory-system performance; even when there is true parallelism in the benchmark, contention in the shared-memory system leads to reduced performance. As a result, the clustered workstations provide higher absolute performance for streaming I/O workloads.

BUCS - a bottom-up cache structure for networked storage servers

“…To Network (2) System RAM (3) (1) Figure 1. Data flows in a traditional system for three cases: (1) A network read request finds data in the system RAM.…”

“…Internal Bus (3) (1) L1 Cache (2) To Network (4) Figure 2. Data flows in a BUCS system for four cases:…”

“…The bus contention problem is not new and was pointed out a few years ago by Arpaci-Dusseau et al [2] for streaming applications. Barve et al [4] found that the overall performance of the storage subsystem is bound by the performance of the I/O bus that connects multiple disks because all disk services are serialized due to bus contention.…”

“…As a result, it has become the major performance bottleneck for high performance servers. Extensive research has been done in addressing this bottleneck problem [2] [4]. Most notable research efforts in this area aim at increasing the bandwidth of system interconnects by replacing PCI with PCI-X, PCI Express, or InfiniBand [1].…”

A Parallel External-Memory Frontier Breadth-First Traversal Algorithm for Clusters of Workstations

“…The use of the sample sorting approach instead of the bucket sorting approach is motivated by the proficiency of sample sorting to deal with large keys and arbitrary key distributions. Whether the sample sorting or the bucket sorting approach is used, performance is limited by the bandwidth available for disk-to-disk data streaming [2].…”