Data Locality Aware Strategy for Two-Phase Collective I/O

Filgueira, Rosa; Singh, David E.; Pichel, Juan C.; Isaila, Florin; Carretero, Jesús

doi:10.1007/978-3-540-92859-1_14

Cited by 10 publications

(11 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The new proposed version of LA-Two-Phase I/O differs from the first [15] in the assignment of aggregators among processes. The first version of LA-Two-Phase I/O Fig.…”

Section: La-two-phase I/omentioning

confidence: 98%

“…In a previous work [15], we performed a study of different algorithms for LAP, and we noticed that the fastest algorithm implemented is the Jonker and Volgenant in all cases, and for this reason we have chosen it for LA-Two-Phase I/O.…”

Section: Dynamic I/o Aggregator Pattern For Two-phase I/omentioning

confidence: 98%

“…In a previous work [15], we have developed first version of LA-Two-Phase I/O technique, where it only reallocated the file domain of each aggregator, without changing the assignment of aggregators. On parallel machines, where each compute nodes contains a multi-core CPU or multiple processors, ROMIO picks by default one of the processes as the aggregator of each node.…”

Section: Two-phase I/omentioning

confidence: 99%

See 2 more Smart Citations

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

et al. 2010

Self Cite

View full text Add to dashboard Cite

This work presents an optimization of MPI communications, called Dynamic-CoMPI, which uses two techniques in order to reduce the impact of communications and non-contiguous I/O requests in parallel applications. These techniques are independent of the application and complementaries to each other. The first technique is an optimization of the Two-Phase collective I/O technique from ROMIO, called Locality aware strategy for Two-Phase I/O (LA-Two-Phase I/O). In order to increase the locality of the file accesses, LA-Two-Phase I/O employs the Linear Assignment Problem (LAP) for finding an optimal I/O data communication schedule. The main purpose of this technique is the reduction of the number of communications involved in the I/O collective operation. The second technique, calledAdaptive-CoMPI, is based on run-time compression of MPI messages exchanged by applications. Both techniques can be applied on every application, because both of them are transparent for the users. Dynamic-CoMPI has been validated by using several MPI benchmarks and real HPC applications. The results show that, for many of the considered scenarios, important reductions in the execution time are achieved by reducing the size and the number of the messages. Additional benefits of our approach are the reduction of the total communication time and the network contention, thus enhancing, not only performance, but also scalability.

show abstract

“…The new proposed version of LA-Two-Phase I/O differs from the first [15] in the assignment of aggregators among processes. The first version of LA-Two-Phase I/O Fig.…”

Section: La-two-phase I/omentioning

confidence: 98%

Section: Dynamic I/o Aggregator Pattern For Two-phase I/omentioning

confidence: 98%

Section: Two-phase I/omentioning

confidence: 99%

See 1 more Smart Citation

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

et al. 2010

Self Cite

View full text Add to dashboard Cite

show abstract

“…Finally, the placement of aggregators is a well-known problem. Certain approaches focus on data locality and a polynomial time assignment algorithm (the Hungarian algorithm) to reduce the communication between compute nodes and aggregators [8]. Others concentrated their efforts on the specific problem of sparse data patterns on BG/Q by offering an algorithm to take paths on the network topology into account [3].…”

Section: Related Workmentioning

confidence: 99%

Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers

Tessier

Malakar²,

Vishwanath³

et al. 2016

2016 First International Workshop on Communication Optimizations in HPC (COMHPC)

Self Cite

View full text Add to dashboard Cite

Reading and writing data efficiently from storage systems is critical for high performance data-centric applications. These I/O systems are being increasingly characterized by complex topologies and deeper memory hierarchies. Effective parallel I/O solutions are needed to scale applications on current and future supercomputers. Data aggregation is an efficient approach consisting of electing some processes in charge of aggregating data from a set of neighbors and writing the aggregated data into storage. Thus, the bandwidth use can be optimized while the contention is reduced. In this work, we take into account the network topology for mapping aggregators and we propose an optimized buffering system in order to reduce the aggregation cost. We validate our approach using micro-benchmarks and the I/O kernel of a large-scale cosmology simulation. We show improvements up to 15× faster for I/O operations compared to a standard implementation of MPI I/O.

show abstract

“…In that context, rather than easing the tuning process, using kernel-based DMA methods is another parameter that changes according to the communication workload. Several efforts have proposed to embrace the hierarchical nature of Grid systems network [9,10,11]. These papers propose different approaches to map the collective communication topology to the actual network topology, an idea that applies as well to multicore processors.…”

Section: Related Workmentioning

confidence: 99%

Locality and Topology Aware Intra-node Communication among Multicore CPUs

Bosilca

Bouteiller

et al. 2010

Recent Advances in the Message Passing Interface

View full text Add to dashboard Cite

Abstract. A major trend in HPC is the escalation toward manycore, where systems are composed of shared memory nodes featuring numerous processing units. Unfortunately, with scale comes complexity, here in the form of non-uniform memory accesses and cache hierarchies. For most HPC applications, harnessing the power of multicores is hindered by the topology oblivious tuning of the MPI library. In this paper, we propose a framework to tune every type of shared memory communications according to locality and topology. An implementation inside Open MPI is evaluated experimentally and demonstrates significant speedups compared to vanilla Open MPI and MPICH2.

show abstract

Data Locality Aware Strategy for Two-Phase Collective I/O

Cited by 10 publications

References 16 publications

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

Topology-Aware Data Aggregation for Intensive I/O on Large-Scale Supercomputers

Locality and Topology Aware Intra-node Communication among Multicore CPUs

Contact Info

Product

Resources

About