Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

Fiore, Sandro; Płóciennik, Marcin; Doutriaux, Charles; Palazzo, Cosimo; Jerome, Boutte, Jason; Żok, Tomasz; Elia, Donatello; Owsiak, M.; D'Anca, Alessandro; Shaheen, Zeshawn; Bruno, Rodrigo; Fargetta, Marco; Caballer, Miguel; Moltó, Germán; Blanquer, Ignácio; Barbera, R.; David, M.; Donvito, Giacinto; Williams, D. N.; Anantharaj, V. G.; Salomoni, Davide; Aloisio, Giovanni

doi:10.1109/bigdata.2016.7840941

Cited by 8 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental projects in the field of data analysis have been implemented in PSNC for a long time [22][23] [24]. On the one hand, these are works connected with statistical analysis of various big data sets and the use of modern technologies of machine learning and artificial intelligence [25].…”

Section: Example 1 -Traffic Analysismentioning

confidence: 99%

A Concept of Innovation Hub for Smart Applications, Enabling Pro-active Approach to Urban Policy and Planning Processes

Mazurek

Stroiński

2020

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

In recent years, the question of how to develop smart applications in environments where solutions already exist has become a challenge for many trends of technological transformation e.g. in industry, transport or smart cities. The latter trend is additionally very sensitive because of the social dimension of the impact of transformation, where the human being should be in the centre of attention. In this paper we present the concept of the Innovation Hub in Poznań, which is a framework for digital transformation focused on integration and development towards proactive digital city system. Starting from the pillars of the digital transformation, i.e. the City Development Strategy and existing wellfunctioning smart city components, we justify the evolutionary model of transformation by creating favorable conditions for the implementation of Proof of Concept projects with the active participation of residents. In addition, we point out examples of projects that were linked to decision-making processes.

show abstract

Section: Example 1 -Traffic Analysismentioning

confidence: 99%

A Concept of Innovation Hub for Smart Applications, Enabling Pro-active Approach to Urban Policy and Planning Processes

Mazurek

Stroiński

2020

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…In the context of the H2020 INDIGO-Datacloud project [21], the Precipitation Trend Analysis (PTA) was selected as a pilot case [22] [23] since it is scientifically relevant and also general enough to validate the infrastructural aspects that also apply to other classes of data analysis (e.g. outlier analysis).…”

Section: B Multi-model Climate Data Analysis: Key Conceptsmentioning

confidence: 99%

“…In previous work [22] [23], a distributed solution based on a two-level workflow approach was proposed. That was the first step towards the Analytics-Hub concept, which was not mature enough at that time.…”

Section: A Architectural View In the Largementioning

confidence: 99%

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Fiore

Elia

Palazzo

et al. 2018

2018 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro-Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

show abstract

“…The approach described here is being used by several user communities that have been engaged within the project [14, 72,20,68,1,19,11,53,41]. The developed solutions have also resulted in community and upstream code contributions to major open source solutions like OpenStack and OpenNebula.…”

Section: Elastic Mesos Clustermentioning

confidence: 99%

Orchestrating Complex Application Architectures in Heterogeneous Clouds

et al. 2017

View full text Add to dashboard Cite

Private cloud infrastructures are now widely deployed and adopted across technology industries and research institutions. Although cloud computing has emerged as a reality, it is now known that a single cloud provider cannot fully satisfy complex user requirements. This has resulted in a growing interest in developing hybrid cloud solutions that bind together distinct and heterogeneous cloud infrastructures. In this paper we describe the orchestration approach for heterogeneous clouds that has been implemented and used within the INDIGO-DataCloud project. This orchestration model uses existing open-source software like OpenStack and leverages the OASIS Topology and Specification for Cloud Applications (TOSCA) open standard as the modeling language. Our approach uses virtual machines and Docker containers in an homogeneous and transparent way providing consistent application deployment for the users. This approach is illustrated by means of two different use cases in different scientific communities, implemented using the INDIGO-DataCloud solutions

show abstract

Distributed and cloud-based multi-model analytics experiments on large volumes of climate change data in the earth system grid federation eco-system

Cited by 8 publications

References 8 publications

A Concept of Innovation Hub for Smart Applications, Enabling Pro-active Approach to Urban Policy and Planning Processes

A Concept of Innovation Hub for Smart Applications, Enabling Pro-active Approach to Urban Policy and Planning Processes

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Orchestrating Complex Application Architectures in Heterogeneous Clouds

Contact Info

Product

Resources

About