Evaluating interconnect and virtualization performance for high performance computing

Ramakrishnan, Lavanya; Canon, R. S.; Muriki, Krishna; Sakrejda, I.; Wright, Nicholas J.

doi:10.1145/2088457.2088459

Cited by 21 publications

(19 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, their work is more focused on the overall performance of the evaluated hypervisors instead of the virtualized network performance and the scalability of communications. Ramakrishnan et al [32] stated that virtualized network is the main performance bottleneck, after analyzing the communication overhead of a number of different interconnect technologies, including 10 Gigabit Ethernet. Furthermore, Zhai et al [33] conducted a comprehensive evaluation of MPI applications on 16 CC1 instances, revealing a significant performance increase compared to previous evaluations on standard and High-CPU EC2 instances.…”

Section: Hpc In the Cloudmentioning

confidence: 99%

Performance analysis of HPC applications in the cloud

Expósito

Taboada

Ramos

et al. 2013

Future Generation Computer Systems

View full text Add to dashboard Cite

The scalability of High Performance Computing (HPC) applications depends heavily on the efficient support of network communications in virtualized environments. However, Infrastructure as a Service (IaaS) providers are more focused on deploying systems with higher computational power interconnected via high-speed networks rather than improving the scalability of the communication middleware. This paper analyzes the main performance bottlenecks in HPC applications scalability on Amazon EC2 Cluster Compute platform: (1) evaluating the communication performance on shared memory and a virtualized 10 Gigabit Ethernet network; (2) assessing the scalability of representative HPC codes, the NAS Parallel Benchmarks, using an important number of cores, up to 512; (3) analyzing the new cluster instances (CC2), both in terms of single instance performance, scalability and costefficiency of its use; (4) suggesting techniques for reducing the impact of the virtualization overhead in the scalability of communication-intensive HPC codes, such as the direct access of the Virtual Machine to the network and reducing the number of processes per instance; and (5) proposing the combination of message-passing with multithreading as the most scalable and cost-effective option for running HPC applications on Amazon EC2 Cluster Compute platform.

show abstract

Section: Hpc In the Cloudmentioning

confidence: 99%

Performance analysis of HPC applications in the cloud

Expósito

Taboada

Ramos

et al. 2013

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…[11] studies how virtualization overhead impacts on performance in High Performance Computing (HPC). The study is based on the use of standard benchmarks for HPC applications, being the main conclusion that virtualization can have, at least, a 60% performance penalty.…”

Section: Related Workmentioning

confidence: 99%

Cost Evaluation of Migrating a Computation Intensive Problem from Clusters to Cloud

Hernández

Fabra

Álvarez

et al. 2013

Economics of Grids, Clouds, Systems, and Services

View full text Add to dashboard Cite

Abstract. Cloud has emerged as an alternative to clusters and grids. Its adoption as an execution environment capable of supporting the high requirements of scientific computations is still an open question. In a previous work, the authors conducted successfully a practical experience of taking advantage of clusters and grids to solve a semantic annotation problem in 178 days. In this work, the authors analyse the cost of solving that problem and compare it with the cost of solving it in a pure Cloud scenario. For this last, a detailed cost estimation is conducted according to the data extracted from the actual execution of a reduced dataset on Amazon EC2. As a result, the suitability of using Cloud-based solutions to solve large and complex scientific problems is discussed.

show abstract

“…This work was accepted for publication in the Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems Workshop (PMBS11) held in conjunction with SC11, Seattle, November 2011 [70] Our benchmarks and other studies [62,23,66,74,46] highlight the performance impact in virtual environments. These studies show that tightly coupled applications perform poorly in virtualized environments such as Amazon EC2.…”

Section: Scaling Study and Interconnect Performancementioning

confidence: 99%

The Magellan Final Report on Cloud Computing

Coghlan¹

2011

View full text Add to dashboard Cite

Executive SummaryThe goal of Magellan, a project funded through the U.S. Department of Energy (DOE) Office of Advanced Scientific Computing Research (ASCR), was to investigate the potential role of cloud computing in addressing the computing needs for the DOE Office of Science (SC), particularly related to serving the needs of midrange computing and future data-intensive computing workloads. A set of research questions was formed to probe various aspects of cloud computing from performance, usability, and cost. To address these questions, a distributed testbed infrastructure was deployed at the Argonne Leadership Computing Facility (ALCF) and the National Energy Research Scientific Computing Center (NERSC). The testbed was designed to be flexible and capable enough to explore a variety of computing models and hardware design points in order to understand the impact for various scientific applications. During the project, the testbed also served as a valuable resource to application scientists. Applications from a diverse set of projects such as MG-RAST (a metagenomics analysis server), the Joint Genome Institute, the STAR experiment at the Relativistic Heavy Ion Collider, and the Laser Interferometer Gravitational Wave Observatory (LIGO), were used by the Magellan project for benchmarking within the cloud, but the project teams were also able to accomplish important production science utilizing the Magellan cloud resources.Cloud computing has garnered significant attention from both industry and research scientists as it has emerged as a potential model to address a broad array of computing needs and requirements such as custom software environments and increased utilization among others. Cloud services, both private and public, have demonstrated the ability to provide a scalable set of services that can be easily and cost-effectively utilized to tackle various enterprise and web workloads. These benefits are a direct result of the definition of cloud computing: on-demand self-service resources that are pooled, can be accessed via a network, and can be elastically adjusted by the user. The pooling of resources across a large user base enables economies of scale, while the ability to easily provision and elastically expand the resources provides flexible capabilities.Following the Executive Summary we summarize the key findings and recommendations of the project. Greater detail is provided in the body of the report. Here we briefly summarize some of the high-level findings from the study.• Cloud approaches provide many advantages, including customized environments that enable users to bring their own software stack and try out new computing environments without significant administration overhead, the ability to quickly surge resources to address larger problems, and the advantages that come from increased economies of scale. Virtualization is the primary strategy of providing these capabilities. Our experience working with application scientists using the cloud demonstrated the power of virtualization to enable ...

show abstract

Evaluating interconnect and virtualization performance for high performance computing

Cited by 21 publications

References 2 publications

Performance analysis of HPC applications in the cloud

Performance analysis of HPC applications in the cloud

Cost Evaluation of Migrating a Computation Intensive Problem from Clusters to Cloud

The Magellan Final Report on Cloud Computing

Contact Info

Product

Resources

About