Topology-Aware Strategy for MPI-IO Operations in Clusters

Liu, Weifeng; Zhou, Jie; Guo, Meng

doi:10.1155/2018/2068490

Cited by 3 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The topology aware algorithm (TA-Two-Phase I/O) takes the I/O pattern and network topology into consideration when assigning file domains to aggregators [17]. It tries to schedule the inter-process communications to the low levels of the tapered hierarchical networks.…”

Section: Topology Aware Algorithm For Two-phase I/o (Ta-two-phase mentioning

confidence: 99%

Topology Aware Algorithm for Two-Phase I/O in Clusters With Tapered Hierarchical Networks

Liu

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

It is important for large scale scientific simulations to read and write data from parallel file system efficiently. To decrease the I/O bottleneck of scientific simulations, many middle-ware solutions had been developed, and the two-phase scheme is one well-known I/O algorithm designed for collective I/O operations. During two-phase I/O based operations, a subset of processes is selected to aggregate noncontiguous pieces of data in the shuffle phase before doing collective reads/writes in the I/O phase. In the meantime, the tapered hierarchical network has long been proposed in order to decrease procurement and power cost. Higher bandwidth and lower latency can be provided in the low levels of tapered hierarchical network. In this paper, we presented a new implementation of two-phase I/O algorithm which takes into consideration the communication pattern and the topology of tapered hierarchical network when scheduling the inter-process communications during the shuffle phase. We validated the new algorithm on our high performance computers and obtained the experimental data on the I/O kernels of some simulations. A significant improvement of the shuffle phase performance was achieved by our new algorithm when compared with the previous two-phase I/O implementations.

show abstract

Section: Topology Aware Algorithm For Two-phase I/o (Ta-two-phase mentioning

confidence: 99%

Topology Aware Algorithm for Two-Phase I/O in Clusters With Tapered Hierarchical Networks

Liu

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…The scope of techniques used will be very huge, including checkpoint-prelaunch variations dependent on uncoordinated replication of rollback recovery, fault tolerance based on algorithms where mathematical properties are leveraged to escape checkpoints Another common characteristic found in most of these advanced methods for recovery of failure, as compared to traditional rollback recovery where the entire machine is used the program is expected to continue running despite processor failures, minimizing the I / O, downtime and overhead losses incurred [6]. The MPI Standard does not, however, Nevertheless, once affected, the MPI Standard does not specify the exact actions of MPI implementations, by failures.…”

Section: Introductionmentioning

confidence: 99%

SDNOFS: Software Defined Networking with Openflow Switches & BCN-ECN with ALTQ for Congestion Avoidance

Sabbi*,

Shrivastava,

Wagh

et al. 2020

IJRTE

View full text Add to dashboard Cite

High-performance computing cluster in a cloud environment. High-performance computing (HPC) helps scientists and researchers to solve complex problems involving multiple computational capabilities. The main reason for using a message passing model is to promote application development, porting, and execution on the variety of parallel computers that can support the paradigm. Since congestion avoidance is critical for the efficient use of different applications, an efficient method for congestion management in software-defined networks based on Open Flow protocol has been presented. This paper proposed two methods; initially, to avoid the congestion problem used by Software Defined Networks (SDN) with open flow switches, this method was originally defined as a communication protocol in SDN environments which allows the SDN controller to interact directly with the forwarding plane of network devices such as switches and routers, both physical and virtual (hypervisorbased), so that it could better adapt to changing business requirements.. Second, to enhance the quality of service and avoid the congestion problem used BCN-ECN with ALTQ. While comparing the existing method, the SDN open flow switches and BCN-ECN with ALTQ provides 98 % accuracy. Usage of these proposed methods will enhance the parameters structures delay time, level of congestion quality time and execution time

show abstract

On Overlapping Communication and File I/O in Collective Write Operation

Feki

Gabriel

2020

2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

View full text Add to dashboard Cite

Topology-Aware Strategy for MPI-IO Operations in Clusters

Cited by 3 publications

References 19 publications

Topology Aware Algorithm for Two-Phase I/O in Clusters With Tapered Hierarchical Networks

Topology Aware Algorithm for Two-Phase I/O in Clusters With Tapered Hierarchical Networks

SDNOFS: Software Defined Networking with Openflow Switches & BCN-ECN with ALTQ for Congestion Avoidance

On Overlapping Communication and File I/O in Collective Write Operation

Contact Info

Product

Resources

About