Experiences Using OpenMP Based on Compiler Directed Software DSM on a PC Cluster

Abstract

In this work we repo t on our experiences running OpenMP programs on a commodity cluster of PCs running a softw_,re distributed shared memory (DSM) system. We describe our test environment and report on the performance of a subset of the NAS Parallel Benchmarks that have been automaticaly parall elized for OpenMP. We compare the performance of the OpenMP implementations with that of tac_r message passing counterparts and discuss performance differences. 1 Introduction Computer Architectures usi_g clusters of PCs with commodity networking have become a low cost alternative for high end scientific computing. Currently message passing is the dominating programruing model for such cluster_. The development of a parallel program based on message passing adds a new level of complexity t_, the software engineering process since not only computation, but also the explicit movement of dala hetween the processes must be specified. Shared memory parallel processors (SMP) provide a user friendlier programming model. The use of globally addressable men_oly allows users to exploit parallelism while avoiding the difficulties of explicit data distribution on _arallel machines. Parallelism is commonly achieved by multi-threading the execution of loops. Comi_iler directives to support multithreaded execution of loops are supported on most shared memory pardi,el platforms. In addition, many compilers provide an automatic parallelization feature taking all tt,e burden of code analysis off the user. Efficiency of compiler parallelized code is often limited, since _ thorough dependence analysis is not possible without user information. Alternatively, there are para tetization support tools available which take the tedious work of dependence analysis and generation of directives off the user but allow user guidance for critical parts of the code. An example Of such a tool is CAPO [10]. While shared memory a 'chitectures provide a convenient programming model for the user, their drawback is that they are e_pensive and the scalability of the code may be limited due to poor data "HLRS (High Performance C(, nputing Center, Stuttgart) t NASA employee of Comput, _ Sciences Corporation. Programming ModelsCurrently message passing md shared address space are the two leading programming models for clusters of SMPs. Message PassingMessage passing is a well uaderstood programming paradigm. The computational work and the associated data are distributed between a number of processes. If a process needs to access data located in the memory of another 1,rocess, it has to be communicated via the exchange of messages. The data transfer requires cooperative operations to be performed by each process, that is, every send must have a matching recetve. The regular message passing communication achieves two effects: communication of data fron sender to receiver and synchronization of sender with receiver. MPI (Message Passing Interface) [12] is a widely accepted standard for writing message passing programs. It is a standard t,rogramming interface for the c...

Running OpenMP applications efficiently on an everything-shared SDSM

“…Although we have not been able to do a direct comparison of our results with other SDSM system, we can tell that similar results were obtained by Müller et al using a relaxed consistency SDSM [7].…”

“…The most significant ones are the OpenMP translator developed by Hu et al [8], OpenMP on the SCASH system [7,14], and ParADE [11].…”

“…OpenMP has also been implemented on top of the SCASH system [7,14]. This approach uses the Omni compiler [15] to transform OpenMP code to parallel code with calls to a runtime library.…”

Scalable RDMA performance in PGAS languages

“…Several combinations of OpenMP runtime plus SDSM systems have been implemented [12]. The most significant ones are the OpenMP translator developed by Hu et al [13] on top of Treadmarks [17], OpenMP on the SCASH system [11], and ParADE [16]. There is also NanosDSM [9] which uses sequential-semantic memory consistency.…”

A package for OpenCL based heterogeneous computing on clusters with many GPU devices

“…One approach uses MPI on top of OpenMP: MPI distributes tasks to cluster-nodes, while OpenMP distributes the tasks further within each node. In the second approach, OpenMP establishes a cluster-wide Distributed Shared-Memory (DSM), which is implemented using MPI [9]. While the second approach is attractive due to OpenMP's ease of programming, its main disadvantage is the complexity and overhead to support DSM in large-scale configurations.…”