Dynamic Resizing of Parallel Scientific Simulations: A Case Study Using LAMMPS

Sudarsan, Rajesh; Ribbens, Calvin J.; Farkas, Diana

doi:10.1007/978-3-642-01970-8_18

Cited by 12 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ReSHAPE [14] integrates job reconfiguration techniques with job scheduling in a framework that also considers the current performance of the execution. Complementary research using this framework analyzes its impact on individual performance and throughput in small workloads [15,16]. That solution, however, requires all applications in the cluster to be specifically-developed to be flexible under the ReSHAPE framework.…”

Section: Related Workmentioning

confidence: 99%

DMR API: Improving cluster productivity by turning applications into malleable

et al. 2018

View full text Add to dashboard Cite

Adaptive workloads can change on-the-fly the configuration of their jobs, in terms of number of processes. In order to carry out these job reconfigurations, we have designed a methodology which enables a job to communicate with the resource manager and, through the runtime, to change its number of MPI ranks. The collaboration between both the workload manager-aware of the queue of jobs and the resource allocation-and the parallel runtime-able to transparently handle the processes and the program data-is crucial for our throughput-aware malleability methodology. Hence, when a job triggers a reconfiguration, the resource manager will check the cluster status and return an action: an expansion, if there are spare resources; a shrink, if queued jobs can be initiated; or none, if no change can improve the global productivity. In this paper, we describe the internals of our framework and how it is capable of reducing the global workload completion time along with providing a smarter usage of the underlying resources. For this purpose, we present a thorough study of the adaptive workloads processing by showing the detailed behavior of our framework in representative experiments and $ This is an extended version of a previous work presented in the ICPP'17 workshop P2S2 [1] $$

show abstract

Section: Related Workmentioning

confidence: 99%

DMR API: Improving cluster productivity by turning applications into malleable

et al. 2018

View full text Add to dashboard Cite

show abstract

“…This is the most complex HPC application that has been turned into malleable so far. The malleability capabilities were added exploiting the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) support for file-based checkpoint and restart (see Sudarsan et al, 2009). A synthetic workload (see Sudarsan and Ribbens, 2016).…”

Section: Related Workmentioning

confidence: 99%

Dynamic reconfiguration of noniterative scientific applications: A case study with HPG aligner

Iserte

Martínez

Barrachina

et al. 2018

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Several studies have proved the benefits of job malleability, that is, the capacity of an application to adapt its parallelism to a dynamically changing number of allocated processors. The most remarkable advantages of executing malleable jobs as part of a high performance computer workload are the throughput increase and the more efficient utilization of the underlying resources. Malleability has been mostly applied to iterative applications where all the processes execute the same operations over different sets of data and with a balanced per process load. Unfortunately, not all scientific applications adhere to this process-level malleable job structure. There are scientific applications which are either noniterative or present an irregular per process load distribution. Unlike many other reconfiguration tools, the Dynamic Management of Resources Application Programming Interface (DMR API) provides the necessary flexibility to make malleable these out-of-target applications. In this article, we study the particular case of using the DMR API to generate a malleable version of HPG aligner, a distributed-memory noniterative genomic sequencer featuring an irregular communication pattern among processes. Through this first conversion of an out-of-target application to a malleable job, we both illustrate how the DMR API may be used to convert this type of applications into malleable and test the benefits of this conversion in production clusters. Our experimental results reveal an important reduction of the malleable HPG aligner jobs completion time compared to the original HPG aligner version. Furthermore, HPG aligner malleable workloads achieve a greater throughput than their fixed counterparts.

show abstract

“…A detailed survey on various application types (Rigid, Moldable, malleable) was conducted in [9]. Resizing application to improve performance has been investigated by many authors, including [16,5,20,19] among others. A related recent study is the design of a MPI prototype for enabling tolerance in Moldable MapReduce applications [11].…”

Section: Moldable and Gridshaped Applicationsmentioning

confidence: 99%

Do Moldable Applications Perform Better on Failure-Prone HPC Platforms?

Fèvre¹,

Bosilca²,

Bouteiller³

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper compares the performance of different approaches to tolerate failures using checkpoint/restart when executed on large-scale failure-prone platforms. We study (i) Rigid applications, which use a constant number of processors throughout execution; (ii) Moldable applications, which can use a different number of processors after each restart following a fail-stop error; and (iii) GridShaped applications, which are moldable applications restricted to use rectangular processor grids (such as many dense linear algebra kernels). For each application type, we compute the optimal number of failures to tolerate before relinquishing the current allocation and waiting until a new resource can be allocated, and we determine the optimal yield that can be achieved. We instantiate our performance model with realistic applicative scenarios and make it publicly available for further usage.

show abstract

Dynamic Resizing of Parallel Scientific Simulations: A Case Study Using LAMMPS

Cited by 12 publications

References 11 publications

DMR API: Improving cluster productivity by turning applications into malleable

DMR API: Improving cluster productivity by turning applications into malleable

Dynamic reconfiguration of noniterative scientific applications: A case study with HPG aligner

Do Moldable Applications Perform Better on Failure-Prone HPC Platforms?

Contact Info

Product

Resources

About