Data Avenue: Remote Storage Resource Management in WS-PGRADE/gUSE

Hajnal, Ákos; Farkas, Zoltán; Kacsuk, Péter

doi:10.1109/iwsg.2014.7

Cited by 7 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a service where the usage dynamically changes is the Data Avenue data staging service [19]. Data Avenue also enables to upload and download files and directories between the user's computer and a data storage (using the storage access protocols mentioned above).…”

Section: Micado Prototype Implementationmentioning

confidence: 99%

MiCADO—Microservice-based Cloud Application-level Dynamic Orchestrator

Kiss

Kacsuk

Kovács

et al. 2019

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

Various scientific and commercial applications require automated scalability and orchestration on cloud computing resources. However, extending applications with such automated scalability on an individual basis is not feasible. This paper investigates how such automated orchestration can be added to cloud applications without major reengineering of the application code. We suggest a generic architecture for an application level cloud orchestration framework, called MiCADO that supports various application scenarios on multiple heterogeneous federated clouds. Besides the generic architecture description, the paper also presents the first MiCADO reference implementation, and explains how the scalability of the Data Avenue service that is applied for data transfer in WS-PGRADE/gUSE based science gateways, can be improved. Performance evaluation of the implemented scalability based on up and downscaling experiments is presented.

show abstract

Section: Micado Prototype Implementationmentioning

confidence: 99%

MiCADO—Microservice-based Cloud Application-level Dynamic Orchestrator

Kiss

Kacsuk

Kovács

et al. 2019

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first MiCADO implementations were primarily focusing on the first application category where the scalability of cloud-based services needs to be automated. Such application example is presented in [6] where the Data Avenue data staging service [22] was successfully extended with MiCADO based automated scalability. Data Avenue enables to transfer files or directories between different data storages having various storage access protocols (e.g.…”

Section: Applications Of Micadomentioning

confidence: 99%

Scalable multi-cloud platform to support industry and scientific applications

Kiss

2018

2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)

View full text Add to dashboard Cite

Cloud computing offers resources on-demand and without large capital investments. As such, it is attractive to many industry and scientific application areas that require large computation and storage facilities. Although Infrastructure as a Service (IaaS) clouds provide elasticity and on demand resource access, the challenges represented by multi-cloud capabilities and application level scalability are still largely unsolved. The CloudSME Simulation Platform (CSSP) extended with the Microservices-based Cloud Application-level Dynamic Orchestrator (MiCADO) addresses such issues. CSSP is a generic multi-cloud access platform for the development and execution of large scale industry and scientific simulations on heterogeneous cloud resources. MiCADO provides application level scalability to optimise execution time and costs. This paper outlines how these technologies have been developed in various European research projects, and showcases several application case-studies from manufacturing, engineering and life-sciences where these tools have been successfully utilised to execute large-scale simulations in an optimised way on heterogeneous cloud infrastructures.

show abstract

“…Data Avenue presented first in is a generic data management component implemented as a web application providing a set of services through a standard web service interface. Synchronous web service operations such as list , mkdir , rmdir , delete , and rename allow for browsing and organizing data files on different storage resources; asynchronous web service operations, copy and move , can be used to transfer files or directories between storage resources (even of different types).…”

Section: Data Bridging Service: Data Avenuementioning

confidence: 99%

“…Our work developed in the European Union Seventh Framework Programme project SCIentific gateway Based User Support (SCI‐BUS) has aimed to significantly improve the state‐of‐the‐art in accessing storages from workflows and gateways: Our solution based on a data bridging service integrated with the WS‐PGRADE/gUSE gateway framework enables the access of many different kinds of storages no matter in which DCI they are located. These storages can be accessed not only statically by the user via a portlet ( portlet‐level access ) but also dynamically by the nodes of a workflow; meanwhile, the workflow is running in several DCIs ( workflow‐level access ). …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Remote storage management in science gateways via data bridging

Hajnal

Márton

Farkas

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYState-of-the-art science gateways can be connected to several distributed computing infrastructures (DCIs) and are able to run jobs and workflows simultaneously in all those DCIs. Flexibility of accessing diverse data storages from these workflows and assisting end users to manage these storages are however the missing features in current gateway implementations, in which these problems often prove to be a barrier of exploiting the power of distributed computing by user communities having no or little IT competence. This paper addresses these issues by integrating a data bridging service called Data Avenue into WS-PGRADE/gUSE portal framework. Data Avenue offers tools for the end users to easily manage their data residing on various storage resources, and also, jobs become capable of accessing different storages regardless of the particular distributed computing infrastructure where the job is currently being run.

show abstract

Data Avenue: Remote Storage Resource Management in WS-PGRADE/gUSE

Cited by 7 publications

References 3 publications

MiCADO—Microservice-based Cloud Application-level Dynamic Orchestrator

MiCADO—Microservice-based Cloud Application-level Dynamic Orchestrator

Scalable multi-cloud platform to support industry and scientific applications

Remote storage management in science gateways via data bridging

Contact Info

Product

Resources

About