An RMS for Non-predictably Evolving Applications

Abstract

Non-predictably evolving applications are applications that change their resource requirements during execution. These applications exist, for example, as a result of using adaptive numeric methods, such as adaptive mesh refinement and adaptive particle methods. Increasing interest is being shown to have such applications acquire resources on the fly. However, current HPC Resource Management Systems (RMSs) only allow a static allocation of resources, which cannot be changed after it started. Therefore, non-predictably evolving applications cannot make efficient use of HPC resources, being forced to make an allocation based on their maximum expected requirements. This paper presents CooRMv2, an RMS which supports efficient scheduling of non-predictably evolving applications. An application can make "pre-allocations" to specify its peak resource usage. The application can then dynamically allocate resources as long as the pre-allocation is not outgrown. Resources which are pre-allocated but not used, can be filled by other applications. Results show that the approach is feasible and leads to a more efficient resource usage.Key-words: resource management; supercomputers; clusters; evolving applications; malleability. Un gestionnaire de ressources pour les applications évolutives Résumé :Les applications évolutives non-prédictibles sont des applications dont les besoins en ressources changent en cours d'exécution. Leurs comportements proviennent, par exemple, des méth-odes numériques adaptatives, comme le raffinement de maillage adaptatif et les méthodes de particules adaptatives. Il y a un intérêt croissant à ce que ce type d'application acquiert des ressources à la volée. Cependant, les gestionnaires de ressources haute performance (HPC) ne supportent qu'une allocation statique des ressources, qui ne peut donc pas changer après le démarrage de l'application. Par conséquence, les applications évolutives non-prédictibles ne peuvent pas utiliser les ressources HPC de manière efficace, car elles sont forcées de faire une demande de ressources en fonction de leurs pics de besoins.Cet article présente CooRMv2, un gestionnaire de ressources qui permet l'ordonnancement efficace d'applications évolutives non-prédictibles. Une application peut faire des « pré-allocations » pour exprimer ses pics de besoins en ressources. Ensuite, l'application peut dynamiquement allouer de manière sûre des ressources tant que la pré-allocation n'est pas dépassée. Les ressources pré-allouées, mais inutilisées, sont à la disposition des autres applications. Les résultats montrent que l'approche est réalisable et que l'utilisation des ressources est améliorée.

Introducing service-level awareness in the cloud

“…Moldable or malleable applications [17] can be directed to trade resources for execution time, for example, an application may run on 4 cores for 1 hour or 2 cores for 2 hours. This additional flexibility in resource management has been shown to bring significant gains [25,27,44,47]. However, the amount of computations essentially remains the same.…”

Combining malleability and I/O control mechanisms to enhance the execution of multiple applications

“…In the context of cloud computing, some solutions for elasticity in I/O have been proposed, such as SpringFS ( Xu et al, 2014 ) as well as solutions based on the Hadoop Distributed File System (HDFS) ( Lim et al, 2010;Cheng et al, 2012 ). Nonetheless, by and large, existing malleability solutions ( Casanova et al, 2014;Kalé et al, 2002;Klein and Pérez, 2011;Hungershofer, 2004;Cirne and Berman, 2002;Prabhakaran et al, 2015 ) are mostly confined to the elasticity of compute and memory allocations.…”

Improving Users' Isolation in IaaS: Virtual Machine Placement with Security Constraints

“…every hours by picking an amount to add/delete randomly. The second one is called AMR, it uses the elasticity model introduced in [35].…”