Understanding Scientific Applications for Cloud Environments

Jha, Shantenu; Luckow, André; Merzky, André; Stamou, Katerina

doi:10.1002/9780470940105.ch13

Cited by 23 publications

(13 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The proposed service layers consist of the following: the software as a service layer (SaaS), the platform as a service (PaaS) layer, and the infrastructure as a service (IaaS) layer. Further, clouds can also be classified according to their deployment model into public and private clouds (for further details, refer to [22]). Azure offers services on the platform as a service layer and thus, generally removes the need to manually manage low-level details as virtual machine configurations, operating system installations and updates, etc.…”

Section: Assessing Azure-system: Abstractions and Applicationsmentioning

confidence: 99%

Application-Level Interoperability Across Grids and Clouds

Jha

Luckow

Merzky

et al. 2011

Grids, Clouds and Virtualization

Self Cite

View full text Add to dashboard Cite

Application-level interoperability is defined as the ability of an application to utilize multiple distributed heterogeneous resources. Such interoperability is becoming increasingly important with increasing volumes of data, multiple sources of data as well as resource types. The primary aim of this chapter is to understand different ways in which application-level interoperability can be provided across distributed infrastructure. We achieve this by (i) using the canonical wordcount application, based on an enhanced version of MapReduce that scales-out across clusters, clouds, and HPC resources, (ii) establishing how SAGA enables the execution of wordcount application using MapReduce and other programming models such as Sphere concurrently, and (iii) demonstrating the scale-out of ensemble-based biomolecular simulations across multiple resources. We show user-level control of the relative placement of compute and data and also provide simple performance measures and analysis of SAGA-MapReduce when using multiple, different, heterogeneous infrastructures concurrently for the same problem instance. Finally, we discuss Azure and some of the system-level abstractions that it provides and show how it is used to support ensemble-based biomolecular simulations.

show abstract

Section: Assessing Azure-system: Abstractions and Applicationsmentioning

confidence: 99%

Application-Level Interoperability Across Grids and Clouds

Jha

Luckow

Merzky

et al. 2011

Grids, Clouds and Virtualization

Self Cite

View full text Add to dashboard Cite

show abstract

“…There is the need for programming systems that can express the hybrid usage modes and associated runtime trade-offs and adaptations, as well as coordination and management infrastructures that can implement them in an efficient and scalable manner. Key issues include decomposing applications, components and workflows, determining and provisioning the appropriate mix of grids/clouds resources, and dynamically scheduling them across the hybrid execution environment while satisfying/balancing multiple, possibly changing objectives for performance, resilience, budgets and so on [18].…”

Section: B Objective Driven Hybrid Usage Of Hpc Grids and Cloudsmentioning

confidence: 99%

Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure

Kim

El-Khamra

Jha

et al. 2010

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

Self Cite

View full text Add to dashboard Cite

Clouds are emerging as an important class of distributed computational resources and are quickly becoming an integral part of production computational infrastructures. An important but oft-neglected question is, what new applications and application capabilities can be supported by clouds as part of a hybrid computational platform? In this paper we use the ensemble Kalman-filter based dynamic application workflow and investigate how clouds can be effectively used as an accelerator to address changing computational requirements as well as changing Quality of Service constraints (e.g., deadlines). Furthermore, we explore how application and system-level adaptivity can be used to improve application performance and achieve a more effective utilization of the hybrid platform. Specifically, we adapt the ensemble Kalman-filter based application formulation (serial versus parallel, different solvers etc.) so as to execute efficiently on a range of different infrastructure (from High Performance Computing grids to clouds that support single core and many-core virtual machines). Our results show that there are performance advantages to be had by supporting application and infrastructure level adaptivity. In general, we find that grid-cloud infrastructure can support novel usage modes, such as deadline-driven scheduling, for applications with tunable characteristics that can adapt to varying resource types.

show abstract

“…For example, the MapReduce programming model along with Hadoop, introduced initially for massive distributed data processing, was explored [ 21 – 23 ]. Also, cloud environments are increasingly becoming popular as a solution for massive data management, processing, and analysis [ 19 , 20 , 24 ]. Previously, SAGA-Pilot-based MapReduce and data parallelization strategies were demonstrated for life science problems, in particular, such as alignment of NGS reads [ 20 , 25 , 26 ].…”

Section: Introductionmentioning

confidence: 99%

Developing eThread Pipeline Using SAGA-Pilot Abstraction for Large-Scale Structural Bioinformatics

Ragothaman

Boddu

Kim

et al. 2014

BioMed Research International

Self Cite

View full text Add to dashboard Cite

While most of computational annotation approaches are sequence-based, threading methods are becoming increasingly attractive because of predicted structural information that could uncover the underlying function. However, threading tools are generally compute-intensive and the number of protein sequences from even small genomes such as prokaryotes is large typically containing many thousands, prohibiting their application as a genome-wide structural systems biology tool. To leverage its utility, we have developed a pipeline for eThread—a meta-threading protein structure modeling tool, that can use computational resources efficiently and effectively. We employ a pilot-based approach that supports seamless data and task-level parallelism and manages large variation in workload and computational requirements. Our scalable pipeline is deployed on Amazon EC2 and can efficiently select resources based upon task requirements. We present runtime analysis to characterize computational complexity of eThread and EC2 infrastructure. Based on results, we suggest a pathway to an optimized solution with respect to metrics such as time-to-solution or cost-to-solution. Our eThread pipeline can scale to support a large number of sequences and is expected to be a viable solution for genome-scale structural bioinformatics and structure-based annotation, particularly, amenable for small genomes such as prokaryotes. The developed pipeline is easily extensible to other types of distributed cyberinfrastructure.

show abstract

Understanding Scientific Applications for Cloud Environments

Cited by 23 publications

References 24 publications

Application-Level Interoperability Across Grids and Clouds

Application-Level Interoperability Across Grids and Clouds

Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure

Developing eThread Pipeline Using SAGA-Pilot Abstraction for Large-Scale Structural Bioinformatics

Contact Info

Product

Resources

About