An Open-source Collaboration Environment for Metagenomics Research

Su, Xiaoquan; Ma, Yongzheng; Yang, Hongwei; Chang, Xingzhi; Nan, Kai; Xu, Jian; Ning, Kang

doi:10.1109/escience.2011.10

Cited by 4 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the rapid growth of e-science, domain scientists increasingly rely on computer systems to conduct their research [5] [16] [23] [26], e.g. cluster, grid and HPC (High Performance Computing) systems.…”

Section: Introductionmentioning

confidence: 99%

An Algorithm for Cost-Effectively Storing Scientific Datasets with Multiple Service Providers in the Cloud

Yuan

Liu

Cui

et al. 2013

2013 IEEE 9th International Conference on E-Science

View full text Add to dashboard Cite

Cloud computing provides scientists a platform that can deploy computation and data intensive applications without infrastructure investment. With excessive cloud resources and a decision support system, large generated datasets can be flexibly 1) stored locally in the current cloud, 2) deleted and regenerated whenever reused or 3) transferred to cheaper cloud service for storage. However, due to the pay-as-you-go model, the total application cost largely depends on the usage of computation, storage and bandwidth resources, hence cutting the cost of cloud-based data storage becomes a big concern for deploying scientific applications in the cloud. In this paper, we propose a novel strategy that can costeffectively store large generated datasets with multiple cloud service providers. The strategy is based on a novel algorithm that finds the trade-off among computation, storage and bandwidth costs in the cloud, which are three key factors for the cost of data storage. Both general (random) simulations conducted with popular cloud service providers' pricing models and three specific case studies on real world scientific applications show that the proposed storage strategy is highly cost effective and practical for runtime utilisation in the cloud. 1 This paper is a significantly extended version of our conference paper [32]. The extensions are from the following aspects: 1) more comprehensive description of the T-CSB algorithm, including the new theorem, figure and algorithm pseudo code; 2) utilisation of the T-CSB algorithm to a localoptimisation based cost-effective and efficient storage strategy; 3) efficiency evaluation of the proposed strategy; 4) case studies of real scientific applications of using the proposed strategy.The remainder of this paper is organised as follows. Section 2 presents a motivating example of scientific application and analyses the research problems. Section 3 introduces some preliminaries and data storage cost model in the cloud. Section 4 presents our novel cost-effective storage strategy in detail. Section 5 describes our experimental results for evaluation. Section 6 discusses the related work. Section 7 summarises our conclusions and points out future work.

show abstract

Section: Introductionmentioning

confidence: 99%

An Algorithm for Cost-Effectively Storing Scientific Datasets with Multiple Service Providers in the Cloud

Yuan

Liu

Cui

et al. 2013

2013 IEEE 9th International Conference on E-Science

View full text Add to dashboard Cite

show abstract

“…Scientific research in various disciplines often involves researchers from different organizations collaborating to conduct analysis, experiments or simulations that are data and compute intensive and with unpredictable resource requirements [15], [16]. These kind applications or tools requires highly dynamic resource allocation method.…”

Section: Problemmentioning

confidence: 99%

CCCORE: Cloud Container for Collaborative Research

Suresh

Rao²

2018

IJECE

View full text Add to dashboard Cite

Cloud-based research collaboration platforms render scalable, secure and inventive environments that enabled academic and scientific researchers to share research data, applications and provide access to high-performance computing resources. Dynamic allocation of resources according to the unpredictable needs of applications used by researchers is a key challenge in collaborative research environments. We propose the design of Cloud Container based Collaborative Research (CCCORE) framework to address dynamic resource provisioning according to the variable workload of compute and data-intensive applications or analysis tools used by researchers. Our proposed approach relies on-demand, customized containerization and comprehensive assessment of resource requirements to achieve optimal resource allocation in a dynamic collaborative research environment. We propose algorithms for dynamic resource allocation problem in a collaborative research environment, which aim to minimize finish time, improve throughput and achieve optimal resource utilization by employing the underutilized residual resources. Keyword: Cloud computing Collaborative research Container Dynamic allocation Finish time Residual resources Throughput Copyright © 2018 Institute of Advanced Engineering and Science.All rights reserved. Corresponding Author:Salini Suresh, Computer Science, Bharathiar University, Coimbatore, India. Email: pnsalinisuresh@gmail.com INTRODUCTIONIn mid-1990's various grid-based cyberinfrastructures or e-infrastructures were constituted that integrated high-speed research networks and middleware services and endorsed researchers for collaborative sharing of distributed resources. These firmly unified science gateways served as resource providers for specialized as well as generic research initiatives [1]. However, restricted interface to the data, domainspecific nature of science gateways did not match the requirement of the researchers outside those domains [2]. With the advent of cloud computing, easy reconfigurable and adaptive Virtual private research environments and science clouds became a preferred alternative to a traditional grid or cluster-based einfrastructures. Cloud-based collaborative research platforms provide the researchers with computing, storage resources required to run their applications, and they can collaborate to share data and application, while he concentrates on his area of research. Cloud platform offers compute environment with the huge set of computing resources much bigger than what an individual research organization can afford. Organizations can scale up, scale down the resources, and pay for it according to the usage. Multitenancy provided by cloud architecture enabled the creation of domain and requirement specific virtual private research environments that expedited researchers for collaboration and sharing of the resources [3]. Several science clouds such as Nectar Research cloud [4] provides the infrastructure to run compute-intensive scientific applications [5], [6]. Even though a substant...

show abstract

Biomedical Research Data Cloud Services with Duckling Collaboration LiBrary (CLB)

Dong

Nan

et al. 2013

2013 IEEE 9th International Conference on E-Science

View full text Add to dashboard Cite

An Open-source Collaboration Environment for Metagenomics Research

Cited by 4 publications

References 24 publications

An Algorithm for Cost-Effectively Storing Scientific Datasets with Multiple Service Providers in the Cloud

An Algorithm for Cost-Effectively Storing Scientific Datasets with Multiple Service Providers in the Cloud

CCCORE: Cloud Container for Collaborative Research

Biomedical Research Data Cloud Services with Duckling Collaboration LiBrary (CLB)

Contact Info

Product

Resources

About