Scalability and communication performance of HPC on Azure Cloud

Hassan, Hanan A.; Mohamed, Shimaa; Sheta, Walaa M.

doi:10.1016/j.eij.2015.11.001

Cited by 24 publications

(17 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hassan et al [65] investigated the performance of Intel MPI Benchmark suite (IMB) and NAS Parallel Benchmarks (NPB) on Microsoft Azure on 16 virtual machines. They were more interested in analyzing scalability of these benchmarks considering different point-to-point communication approaches using both MPICH and OpenMPI.…”

Section: Viability: Performance and Cost Concernsmentioning

confidence: 99%

See 1 more Smart Citation

HPC Cloud for Scientific and Business Applications

et al. 2018

View full text Add to dashboard Cite

High Performance Computing (HPC) clouds are becoming an alternative to on-premise clusters for executing scientific applications and business analytics services. Most research efforts in HPC cloud aim to understand the cost-benefit of moving resource-intensive applications from on-premise environments to public cloud platforms. Industry trends show hybrid environments are the natural path to get the best of the on-premise and cloud resources-steady (and sensitive) workloads can run on on-premise resources and peak demand can leverage remote resources in a pay-as-you-go manner. Nevertheless, there are plenty of questions to be answered in HPC cloud, which range from how to extract the best performance of an unknown underlying platform to what services are essential to make its usage easier. Moreover, the discussion on the right pricing and contractual models to fit small and large users is relevant for the sustainability of HPC clouds. This paper brings a survey and taxonomy of efforts in HPC cloud and a vision on what we believe is ahead of us, including a set of research challenges that, once tackled, can help advance businesses and scientific discoveries. This becomes particularly relevant due to the fast increasing wave of new HPC applications coming from big data and artificial intelligence. users have no visibility or concerns on costs of running jobs. However, large clusters do incur expenses and, when not properly managed, can generate resource wastage and poor quality of service.Motivated by the different utilization levels of clusters around the globe and by the need to run even larger parallel programs, in the early 2000s, Grid Computing became relevant for the HPC community. Grids offer users access to powerful resources managed by autonomous administrative domains [50,51]. The notion of monetary costs for running applications was soft, favoring a more collaborative model of resource sharing. Therefore, quality of service was not strict in Grids, having users relying on best-effort policies to run applications.In the late 2000s, cloud computing [8,26,91] was quickly increasing its maturity level and popularity, and studies started to emerge on the viability of executing HPC applications on remote cloud resources. These applications, which consume more resources than traditional cloud applications and usually are executed in batches rather than 24x7 services, range from parallel applications written in Message Passing Interface (MPI) [58,59] to the newest big data [11,14,39,101] and artificial intelligence applications-the latter mostly relying on deep learning [34,80]. Cloud then came up as an evolution of a series of technologies, mainly on virtualization and computer networks, which facilitated both workload management and interaction with remote resources respectively. Apart from software and hardware, cloud offers a business model where users pay for resources on demand. Compared to traditional HPC environments, in clouds users can quickly adjust their resource pools, via a mechanism known as elast...

show abstract

Section: Viability: Performance and Cost Concernsmentioning

confidence: 99%

“…Resources [41,45,60,65,92,105,123] Related to how different resource types impact performance of the applications. Network virtualization and hardware heterogeneity are main causes of poor performance of HPC in cloud; single-node performance is comparable between environments; improvements in infrastructure affect HPC applications positively.…”

Section: Aspectmentioning

confidence: 99%

HPC Cloud for Scientific and Business Applications

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Some have mainly contributed to understanding the cost-benefits of using cloud over on-premise clusters [10,23]. Other aimed at evaluating the performance gap between cloud and on-premise resources [12,20,28]. Additionally, a lot of effort has been invested into the HPC job scheduling, as well as application testing and tuning within the cloud environments [5,6,15,19,30].…”

Section: Introductionmentioning

confidence: 99%

“…Other aimed at evaluating the performance gap between cloud and on-premise resources [12,20,28]. Additionally, a lot of effort has been invested into the HPC job scheduling, as well as application testing and tuning within the cloud environments [5,6,15,19,30]. Therefore, it could be found that most interests of combining HPC and cloud are laid on using cloud as IaaS (infrastructure as a Service).…”

Section: Introductionmentioning

confidence: 99%

Facilitating HPC Operation and Administration via Cloud

Sha¹,

Zhang²,

An³

et al. 2019

JSFI

View full text Add to dashboard Cite

Experiencing a tremendous growth, Cloud Computing offers a number of advantages over other distributed platforms. Introducing the advantages of High Performance Computing (HPC) also brought forward the development of HPCaaS (HPC as a Service), which has mainly focused on flexible access to resources, cost-effectiveness, and the no-maintenance-needed for end-users. Besides providing and using HPCaaS, HPC centers could leverage more from Cloud Computing technology, for instance to facilitate operation and administration of deployed HPC systems, commonly faced by most supercomputer centers. This paper reports the product, EasyOP, developed to realize the idea that one or more Cloud or HPC facilities can be run over a centralized and unified control platform. The main purpose of EasyOP is that the information of HPC systems hardware and system software, failure alarms, jobs scheduling, etc. is sent to the Wuxi cloud computing center. After a series of analysis and processing, we are able to share many valuable data, including alarm and job scheduling status, to HPC users through SMS, email, and WeChat. More importantly, with the data accumulated on the cloud computing center, EasyOP can offer several easy-to-use functions, such as user(s) management, monthly/yearly reports, one-screen monitoring and so on. By the end of 2016, EasyOP successfully served more than 50 HPC systems with almost 10000 nodes and over of 300 regular users.

show abstract

“…Experiments were conducted by executing the hybrid and openMP versions of NAS parallel benchmarks (NPB) at the KNL partition of Tarus, Technical University -Dresden, Germany. NPB are, in general, widely utilised by several researchers for analysing the performance of applications and architectures (Rosales et al, 2016;Hassan et al, 2016;Narayana et al, 2017). The remaining sections of the paper contain the following: Section 2 discusses previous works; Section 3 describes the KNL architecture; Section 4 describes the supportive performance efficiency features of applications on KNL; Section 5 illustrates the findings towards vectorisation, manycore features, and data locality options for the NPB and Section 6 presents a few outlooks and conclusions.…”

Section: Introductionmentioning

confidence: 99%

Performance Improvement Options of Scientific Applications on XeonPhi KNL Architectures

Benedict

2018

IJKEDM

View full text Add to dashboard Cite

Intel's recent manycore processor KNights Landing (KNL) promises high performance for scientific applications. Careful tuning for the complex chip architecture is required to efficiently exploit the chip's hardware resources. This paper describes performance improvement techniques and demonstrates their effectiveness for scientific applications. Experiments were conducted with some of the National Aeronautics and Space Administration (NASA's) advanced supercomputing (NAS) parallel benchmarks, and the effectiveness of: 1) advanced vector extensions (AVX-512) vectorisation support; 2) manycore threading support; 3) the utilisation of thread affinities for different KNL modes, was analysed.

show abstract

Scalability and communication performance of HPC on Azure Cloud

Cited by 24 publications

References 5 publications

HPC Cloud for Scientific and Business Applications

HPC Cloud for Scientific and Business Applications

Facilitating HPC Operation and Administration via Cloud

Performance Improvement Options of Scientific Applications on XeonPhi KNL Architectures

Contact Info

Product

Resources

About