Cloud infrastructure for HPC investment analysis

Santos, Maicon Anca dos; Cavalheiro, Gerson Geraldo H.

doi:10.22456/2175-2745.106794

Cited by 4 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most cases, these companies do not require low-level knowledge of the underlying hardware or high-speed communication between nodes, both of which are typical requirements for HPC workload. 14 Instead, for most enterprise applications, the primary requirements are resiliency and fault tolerance, which can be achieved through redundancy: replicas of the same service are executed in different geographical locations to avoid single points of failure. The HPC use case, however, tends to be the opposite as nodes are allocated as close as possible to each other to reduce communication latency.…”

Section: Challenges For Cloud and Hpc Convergencementioning

confidence: 99%

“…As a result, most companies use cloud resources to ensure service availability and disaster recovery capabilities for their enterprise systems. In most cases, these companies do not require low‐level knowledge of the underlying hardware or high‐speed communication between nodes, both of which are typical requirements for HPC workload 14 . Instead, for most enterprise applications, the primary requirements are resiliency and fault tolerance, which can be achieved through redundancy: replicas of the same service are executed in different geographical locations to avoid single points of failure.…”

Section: Background and Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Enabling the execution of HPC applications on public clouds with HPC@Cloud toolkit

Munhoz,

Castro

2023

Concurrency and Computation

View full text Add to dashboard Cite

The advent of cloud computing has made access to computing infrastructure available to millions of users that face resource constraints. In the context of high performance computing (HPC), public cloud resources have emerged as a cost‐effective alternative to expensive on‐premises clusters. However, there are several challenges and limitations in adopting this approach. This paper proposes HPC@Cloud , a provider‐agnostic open‐source software toolkit that facilitates the migration, testing, and execution of HPC applications in public clouds. The toolkit takes advantage of various fault tolerance technologies to enable the use of inexpensive transient cloud infrastructure, commonly known as “spot” instances. Also, it features integration with singularity containers, allowing users to run complex applications on virtual HPC clusters in a portable and reproducible way. Finally, it provides a data‐based empirical approach to estimating cloud infrastructure costs for HPC workloads. The results obtained on two public cloud providers (AWS and Vultr) show that: (i) HPC@Cloud can efficiently build virtual HPC clusters on the cloud; (ii) the new adaptive fault tolerance strategy outperforms other existing strategies based on blocking restoration; (iii) the integration of singularity containers into HPC@Cloud improves the portability of HPC applications to public clouds with negligible performance penalty to the applications; (iv) the proposed cost prediction approach can estimate the cost of running the applications on AWS and Vultr with up to 93% accuracy on average.

show abstract

Section: Challenges For Cloud and Hpc Convergencementioning

confidence: 99%

Section: Background and Motivationmentioning

confidence: 99%

Enabling the execution of HPC applications on public clouds with HPC@Cloud toolkit

Munhoz,

Castro

2023

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…As such, most companies use cloud resources to guarantee service availability and disaster recovery capabilities for their enterprise systems. In most common cases, these companies neither require low-level knowledge about the underlying hardware nor need high-speed communication between nodes, which are typical requirements for HPC workloads [Santos and Cavalheiro 2020].…”

Section: Challenges For Cloud and Hpc Convergencementioning

confidence: 99%

HPC@Cloud: A Provider-Agnostic Software Framework for Enabling HPC in Public Cloud Platforms

Munhoz

Castro²

2022

Anais Do XXIII Simpósio Em Sistemas Computacionais De Alto Desempenho (SSCAD 2022)

View full text Add to dashboard Cite

The cloud computing paradigm democratized compute infrastructure access to millions of resource-strained organizations, applying economics of scale to massively reduce infrastructure costs. In the High Performance Computing (HPC) context, the benefits of using public cloud resources make it an attractive alternative to expensive on-premises clusters, however there are several challenges and limitations. In this paper, we present HPC@Cloud: a provideragnostic software framework that comprises a set of key software tools to assist in the migration, test and execution of HPC applications in public clouds. HPC@Cloud allows the HPC community to benefit from readily available public cloud resources with minimum efforts and features an empirical approach for estimating cloud infrastructure costs for HPC workloads. We also provide an experimental analysis of HPC@Cloud on two public clouds: Amazon AWS and Vultr Cloud.

show abstract

Strategies for Fault-Tolerant Tightly-Coupled HPC Workloads Running on Low-Budget Spot Cloud Infrastructures

Munhoz

Castro

Mendizabal

2022

2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

View full text Add to dashboard Cite

Cloud infrastructure for HPC investment analysis

Cited by 4 publications

References 24 publications

Enabling the execution of HPC applications on public clouds with HPC@Cloud toolkit

Enabling the execution of HPC applications on public clouds with HPC@Cloud toolkit

HPC@Cloud: A Provider-Agnostic Software Framework for Enabling HPC in Public Cloud Platforms

Strategies for Fault-Tolerant Tightly-Coupled HPC Workloads Running on Low-Budget Spot Cloud Infrastructures

Contact Info

Product

Resources

About