Managing Complexity in Distributed Data Life Cycles Enhancing Scientific Discovery

Grunzke, Richard; Aguilera, Alvaro; Nagel, Wolfgang E.; Krüger, Jens; Herres‐Pawlis, Sonja; Hoffmann, Alexander; Gesing, Sandra

doi:10.1109/escience.2015.72

Cited by 11 publications

(17 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We surveyed most DLC models found in the literature and realized that each only model addressed some specific data stages, such as data discovery [5], data quality [6,9], data sharing [17] and data security [18]. In addition, we also found that each DLC model was proposed to address the particular requirements for a particular scientific field, for example, library and information science in [19], geomorphology in [1920 ecology in [7,8], or to address specific scenarios, such as Smart Cities in [21,22], or support of sciences in [4,10].…”

Section: Related Workmentioning

confidence: 99%

“…DLC models define the sequence of phases in the data life, specify the management policies for each phase, and describe the relationship among phases [6]. Furthermore, a DLC model is designed specifically for a particular field and scenario, addressing its private requirements and challenges [5,8]. The benefits of designing and implementing a DLC model are the following:…”

Section: Introductionmentioning

confidence: 99%

“…Data can then be used in different forms during big data processing (reading, writing, transforming or removing), and then be reused in following processes, therefore drawing the life cycle of data. An appropriate management and organization of diverse and sophisticated data sets during their entire life cycle, including data generation, data acquisition, data preservation, or data processing, becomes a complex and challenging task [4,5]. The main objective of data management is to provide easy, efficient and safe access to data sources and repositories, in order to be able to extract any form of value through complex computing and analytical processes over big data sources.…”

Section: Introductionmentioning

confidence: 99%

“…For this reason, efficient data management and organization systems are a key topic for an effective data to value generation. Data LifeCycle (DLC) models have recently been proposed as an effective data management solution that facilitates the organization of data and the extraction of knowledge in complex data systems [4][5][6][7][8]. DLC models define the sequence of phases in the data life, specify the management policies for each phase, and describe the relationship among phases [6].…”

Section: Introductionmentioning

confidence: 99%

“…• Easing for planning and handling complexity of data management throughout all data life stages [6,7]; • Preparing data products for end-users access, achieving the expected constraints and efficiency requirements [5][6][7]; • Providing high data quality level, removing any kind of waste and noise [6]; • Identifying the appropriate sequence of essential activities related to data life [6]; and • Helping system designers to create sustainable and efficient software [9,10]. In a previous work [11], we surveyed most DLC found in the literature, and evaluated qualitatively each of them with respect to the 6 Vs challenges, namely Value, Volume, Variety, Velocity, Variability and Veracity.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Towards a comprehensive data lifecycle model for big data environments

Sinaeepourfard

García

Masip-Bruin

et al. 2016

Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies

View full text Add to dashboard Cite

A huge amount of data is constantly being produced in the world. Data coming from the IoT, from scientific simulations, or from any other field of the eScience, are accumulated over historical data sets and set up the seed for future Big Data processing, with the final goal to generate added value and discover knowledge. In such computing processes, data are the main resource; however, organizing and managing data during their entire life cycle becomes a complex research topic. As part of this, Data LifeCycle (DLC) models have been proposed to efficiently organize large and complex data sets, from creation to consumption, in any field, and any scale, for an effective data usage and big data exploitation.Several DLC frameworks can be found in the literature, each one defined for specific environments and scenarios. However, we realized that there is no global and comprehensive DLC model to be easily adapted to different scientific areas. For this reason, in this paper we describe the Comprehensive Scenario Agnostic Data LifeCycle (COSA-DLC) model, a DLC model which: i) is proved to be comprehensive as it addresses the 6Vs challenges (namely Value, Volume, Variety, Velocity, Variability and Veracity; and ii), it can be easily adapted to any particular scenario and, therefore, fit the requirements of a specific scientific field. In this paper we also include two use cases to illustrate the ease of the adaptation in different scenarios. We conclude that the comprehensive scenario agnostic DLC model provides several advantages, such as facilitating global data management, organization and integration, easing the adaptation to any kind of scenario, guaranteeing good data quality levels and, therefore, saving design time and efforts for the scientific and industrial communities. CCS Concepts• Information systems➝Data management systems • Computer systems organization➝Architectures➝Other architectures➝ Data flow architectures • Software and its engineering➝ Software system structures➝ Data flow architectures

show abstract