An Empirical Study of SDN-accelerated HPC Infrastructure for Scientific Research

Date, Susumu; Abe, Hirotake; Khureltulga, Dashdavaa; Takahashi, Keichi; Kido, Yoshiyuki; Watashiba, Yasuhiro; U-chupala, Pongsakorn; Ichikawa, Kohei; Yamanaka, Hiroaki; Kawai, Eiji; Shimojo, Shinsuke

doi:10.1109/icccri.2015.13

Cited by 6 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…No parallel applications are considered in their study. The authors in [17] and [18] study the performance impact of OpenFlow (a well-known SDN implementation) on the Message Passing Interface (MPI) collective communication. SDN is proposed to support the dynamic nature of the virtual servers and networks.…”

Section: Related Workmentioning

confidence: 99%

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

et al. 2020

View full text Add to dashboard Cite

The main goal of oil reservoir management is to provide more efficient, price-effective and environmentally more secure oil production. Oil production management includes an accurate characterization of the reservoir and strategies that involve interactions between reservoir data and human assessment. Hence, it is important to graphically visualize and handle massive data sets of oil and gas pressure / saturation levels to help decision makers in statistical analysis, history matching and recovery of hydrocarbons of the reservoir. In this article, we experimentally study the parallelization of intensive computation for a 3-D (three dimensional) oil reservoir data visualization tool. For this tool, we develop and implement a transformation and lighting model to visualize and react with the grid. Herein, we propose a hybrid (shared memory and distributed memory) parallelization technique to adapt with the data processing scalability. We tested these implementations on OpenStack Cloud Virtual Cluster. Our results indicate that although the virtual platform adds overhead for running parallel implementations, utilizing knowledge of the VM location on the compute host and network traffic among VMs to deploy the virtual environment can achieve significant performance enhancements. Hybrid parallel implementation using large data size can achieve 70× speedup over serial execution without owning a costly HPC infrastructure as the conventional parallel processing deployment model.INDEX TERMS Cloud computing, data visualization tool, HPC, hybrid (distributed/shared)-memoryparallel programming, MPI, multi-threading, OpenStack.

show abstract

Section: Related Workmentioning

confidence: 99%

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

et al. 2020

View full text Add to dashboard Cite

show abstract

Section: ) Job Throughputmentioning

confidence: 99%

“…The idea of using dynamic traffic control on the interconnect of HPC cluster systems is shared in [21]- [25]. In [21], an SDN-accelerated HPC cluster system has been proposed as a concept to integrate SDN into HPC cluster systems. Based on this concept, an SDN-enhanced job management system (JMS) [22], [23], which is a job scheduler to manage not only the computing nodes but also the interconnect to improve the job throughput, and SDN-enhanced MPI [24], [25], which enhances MPI protocol using SDN to improve the performance of inter-node communication, have been proposed.…”

Section: ) Job Throughputmentioning

confidence: 99%

Dynamic Traffic Control of Staging Traffic on the Interconnect of the HPC Cluster System

et al. 2020

Self Cite

View full text Add to dashboard Cite

High-performance computing (HPC) cluster systems sometimes adopt a two-layered file system composed of local and global file systems to achieve both capacity and performance in storage. In such a cluster system, the input data of an application needs to be staged from the global storage into the local storage, and the output data needs to be staged from the local storage out to the global storage. This staging operation must be efficiently and quickly performed to gain higher job throughput because an inefficient staging operation prevents waiting job requests from being executed. In particular, in the case of the cluster system with the oversubscribed interconnect shared by the storage and the computing nodes, the inter-node communication and this staging operation traffic collides, which may degrade the job throughput. In this research, we focus on the traffic collision of the inter-node communication and the staging traffic to improve job throughput, targeting the cluster system with the oversubscribed interconnect where these two types of traffic flow. In other words, whether the dynamic control of the traffic flow derived from the staging operation leads to the improvement in the job throughput or not is investigated. For the investigation, we present a traffic collision avoidance method to dynamically configure a set of data paths for each type of the traffic only while the staging operation is conducted. The evaluation in this paper shows that the proposed method avoids a traffic collision and accelerates the staging operation by 22.0% on our cluster system. Also, this evaluation indicates the overhead of the application incurred by the proposed method is negligible. Furthermore, 8.7% of the job execution time is reduced by the proposed method.

show abstract

“…Date et at. [7] have discussed the integration of SDN with highperformance computing (HPC) infrastructure, reporting some specific HPC proposals for dealing with computing relocation, bandwidth, and latency. Finally, in a previous work [19], we studied the impacts of the SDN programmability on SciApps.…”

Section: Background and Related Workmentioning

confidence: 99%

Softening Up the Network for Scientific Applications

Trois

Bona

Fabro

et al. 2017

2017 25th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

View full text Add to dashboard Cite

An Empirical Study of SDN-accelerated HPC Infrastructure for Scientific Research

Cited by 6 publications

References 15 publications

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

Dynamic Traffic Control of Staging Traffic on the Interconnect of the HPC Cluster System

Softening Up the Network for Scientific Applications

Contact Info

Product

Resources

About