One-sided communication on the myrinet-based SMP clusters using the GM message-passing library

Nieplocha, Jarek; Ju, Jialin; Aprà, Edoardo

doi:10.1109/ipdps.2001.925158

Cited by 7 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [3] have shown the performance of QsNet under different communication patterns. Numerous research studies have been done on the Myrinet including [2], [14], and [28]. In [28], the authors have compared the performance of their MPI implementation on top of the Mellanox's InfiniBand, with the MPI implementation over Quadrics QsNet, and the Myrinet D-card.…”

Section: Related Workmentioning

confidence: 99%

Performance evaluation of the Sun Fire Link SMP clusters

Qian

Afsahi

Fredrickson

et al. 2006

IJHPCN

View full text Add to dashboard Cite

As symmetric multiprocessors become commonplace, the interconnection networks and the communication system software in clusters of multiprocessors become critical to achieving high performance. Recently, Sun Microsystems has introduced a new system area network, Sun Fire Link interconnect, for its Sun Fire cluster systems. Sun Fire Link is a memory-based interconnect, where Sun MPI uses the Remote Shared Memory (RSM) model for its user-level internode messaging protocol. In this paper, we present the overall architecture of the Sun Fire Link interconnect, explain how communication is done under the RSM, and describe the Sun MPI implementation. We provide an in-depth performance evaluation of the Sun Fire Link interconnect cluster of four Sun Fire 6800s at the RSM layer, MPI microbenchmark layer, and the application layer. Our results include the performance of the Remote Shared Memory API primitives, MPI overhead on top of the RSM, latency and bandwidth under different communication modes, parameters of the LogP model, collective communications, permutation communications, and two applications (Sphot and SMG2000) from the ASCI Purple suite under the MPI, OpenMP, and MPI-OpenMP programming paradigms. Our results indicate that put has a better performance than get on this interconnect. The Sun MPI implementation achieves an off-node latency of up to 5 microseconds. This is comparable to the Quadrics (QsNet II), and Myrinet (E-card), and better than the InfiniBand. The unidirectional and bidirectional bandwidths are 695 MB/s, and 660 MB/s, respectively. The QsNet II, and InfiniBand achieve higher bandwidths. The LogP parameters indicate the network interface is less capable of offloading the host CPU when the message size increases. The performance of our applications under MPI is better than the OpenMP version, and equal or slightly better than the mixedmode. In general, the Sun Fire Link cluster performs relatively well in most cases. I. INTRODUCTION Clusters of Symmetric Multiprocessors (SMP) have been regarded as viable scalable architectures to achieve supercomputing performance and to meet the ever-increasing demand for high-performance computing. There are two main components in such systems: the SMP node, and the communication subsystem including the interconnect, and the communication system software. Considerable work has gone into the design of SMP systems, and several vendors such as IBM, Sun, Compaq, SGI, and HP offer small to large scale shared memory systems. Sun Microsystems has introduced its Sun Fire systems in three categories of small, midsize, and large SMPs, supporting two to 106 processors, backed up with its Sun Fireplane interconnect [1] used inside the Sun UltraSPARC III Cu. The Sun Fireplane interconnect uses one to four levels of interconnect ASICs to provide better sharedmemory performance. All Sun Fire systems use point-to-point signals with a crossbar rather than a data bus. The interconnection network hardware and the communication system software are the keys to the performance of cl...

show abstract

Section: Related Workmentioning

confidence: 99%

Performance evaluation of the Sun Fire Link SMP clusters

Qian

Afsahi

Fredrickson

et al. 2006

IJHPCN

View full text Add to dashboard Cite

show abstract

“…For high performance applications, researchers have derived benefits from the concurrency and asynchrony of execution of host versus NI tasks and from the low-latency access most NIs have to the actual network transceiver [45], [48], [37]. For media scheduling, we next evaluate the ability of CoProcessors to independently stream media frames to clients once the host CPU has identified the appropriate media files and their storage or remote sources.…”

Section: There Are Multiple Alternatives In Media Streaming From Ni-amentioning

confidence: 99%

“…A property of CoProcessors is that the richer their functionality, the larger the latencies of their message transfers and, possibly, the smaller the total message throughput they support (see Section 3.2) [54], [37]. Our approach to this problem is to specialize NIs such that each NI only performs a limited number of tasks.…”

Section: Nis Can Stream Data From Their Attached Disks With Performanmentioning

confidence: 99%

“…In part, such work is motivated by the presence of programmable CoProcessors in NIs. More importantly, past research has demonstrated improved performance derived from NI reprogramming for control operations that touch multiple machines like collective communications [53], transactions [48], barriers [37], [45], and simpler cluster-wide synchronization constructs [48]. NIs have also been programmed to provide new services, like performance monitoring [34] or like the implementation of portions of protocol stacks on NIs [12].…”

Section: Communication Coprocessorsmentioning

confidence: 99%

See 1 more Smart Citation

On network CoProcessors for scalable, predictable media services

Krishnamurthy

Schwan

West³

et al. 2003

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

This paper presents the embedded realization and experimental evaluation of a media stream scheduler on Network Interface (NI) CoProcessor boards. When using media frames as scheduling units, the scheduler is able to operate in real-time on streams traversing the CoProcessor, resulting in its ability to stream video to remote clients at real-time rates. The contributions of this paper are its detailed evaluation of the effects of placing application or kernel-level functionality, like packet scheduling on NIs, rather than the host machines to which they are attached. The main benefits of such placement are 1) that traffic is eliminated from the host bus and memory subsystem, thereby allowing increased host CPU utilization for other tasks, and 2) that NI-based scheduling is immune to host-CPU loading, unlike host-based media schedulers that are easily affected even by transient load conditions. An outcome of this work is a proposed cluster architecture for building scalable media servers by distributing schedulers and media stream producers across the multiple NIs used by a single server and by clustering a number of such servers using commodity network hardware and software.

show abstract

“…To support the full set of remote memory operations on a cluster of workstations using GM [12], ARMCI uses a client-server architecture [15]. Figure 1 shows the client architecture on a cluster of SMP nodes.…”

Section: Armci -Aggregate Remote Memory Copy Interfacementioning

confidence: 99%

Optimizing synchronization operations for remote memory communication systems

Buntinas

Saify

Panda

et al.

Proceedings International Parallel and Distributed Processing Symposium

Self Cite

View full text Add to dashboard Cite

Synchronization operations, such as fence and locking, are used in many parallel operations accessing shared memory. However, a process which is blocked waiting for a fence operation to complete, or for a lock to be acquired, cannot perform useful computation. It is therefore critical that these operations be implemented as efficiently as possible to reduce the time a process waits idle. These operations also impact the scalability of the overall system. As system sizes get larger, the number of processes potentially requesting a lock increases. In this paper we describe the design and implementation of an optimized operation which combines a global fence operation and a barrier synchronization operation. We also describe our implementation of an optimized lock algorithm. The optimizations have been incorporated into the ARMCI communication library. The global fence and barrier operation gives a factor of improvement of up to 9 over the current implementation in a 16 node system, while the optimized lock implementation gives up to 1.25 factor of improvement. These optimizations allow for more efficient and scalable applications.

show abstract

One-sided communication on the myrinet-based SMP clusters using the GM message-passing library

Cited by 7 publications

References 12 publications

Performance evaluation of the Sun Fire Link SMP clusters

Performance evaluation of the Sun Fire Link SMP clusters

On network CoProcessors for scalable, predictable media services

Optimizing synchronization operations for remote memory communication systems

Contact Info

Product

Resources

About