Context-aware data quality assessment for big data

Ardagna, Danilo; Cappiello, Cinzia; Samá, Walter; Vitali, Monica

doi:10.1016/j.future.2018.07.014

Cited by 93 publications

(47 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For structured data, data quality literature offers several contributions that propose assessment algorithms for these consolidated dimensions, but big data pose new challenges related their main characteristics: volume, velocity and variety. In particular, in order to address volume and velocity issues, it is necessary to redesign assessment methods for exploiting parallel computing scenarios and for reducing the computation space (Ardagna et al, 2018).…”

Section: Data Quality In Big Data Systemsmentioning

confidence: 99%

Big Data Quality Dimensions: A Systematic Literature Review

Ramasamy

Chowdhury

2020

JISTEM USP

View full text Add to dashboard Cite

Although big data has become an integral part of businesses and society, there is still concern about the quality aspects of big data. Past research has focused on identifying various dimensions of big data. However, the research is scattered and there is a need to synthesize the ever involving phenomenon of big data. This research aims at providing a systematic literature review of the quality dimension of big data. Based on a review of 17 articles from academic research, we have presented a set of key quality dimensions of big data.

show abstract

Section: Data Quality In Big Data Systemsmentioning

confidence: 99%

Big Data Quality Dimensions: A Systematic Literature Review

Ramasamy

Chowdhury

2020

JISTEM USP

View full text Add to dashboard Cite

show abstract

“…The authors examine a case study from the automotive industry using the linked enterprise data approach to integrate data from different development tools (Gürdür, Elkhoury and Nyberg, 2018). Ardagna et al (2018) propose a methodology to build a DQ adapter module, which selects the best configuration for a context-aware DQ assessment based on the user main requirements: time minimization, confidence maximization, and budget minimization.…”

Section: Assessment Of Data Qualitymentioning

confidence: 99%

Methodological Assessment of Data Suitability for Defect Prediction

et al. 2020

View full text Add to dashboard Cite

Purpose: This paper provides a domain specific concept to assess data suitability of various data sources along the production chain for defect prediction.Methodology/Approach: A seven-phase methodology is developed in which the data suitability for defect prediction in interlinked production steps is assessed. For this purpose, the manufacturing process is mapped and potential influencing variables on the origin of defects are identified. The available data is evaluated and quantified with regard to the criteria relevancy, completeness, appropriate amount of data, accessibility and interpretability. The individual assessments are then visualized in an overview, gaps in data acquisition are identified and needs for action are derived.Findings: The research shows a seven-phase methodology to systematically assess data suitability for defect prediction and identify data gaps in interlinked production steps.Research Limitation/implication: This research is limited to the analysis of contextual data quality for the use case of defect prediction. Other data analytics applications or processes outside of manufacturing are not included.Originality/Value of paper: The paper provides a new approach to identify gaps in data acquisition by systematically assessing data suitability for defect prediction and deducting needs for action. The accuracy of predictive defect models is then to be improved by the subsequent optimization of the data basis.

show abstract

“…Data quality management techniques used in traditional databases are not enough to handle a big data scenario with heterogeneous sources, which instead requires an "adaptive approach able to trigger the suitable quality assessment methods on the basis of the data type and context in which data have to be used" [4]. Among other challenges, this requires context-dependent quality assessment, which takes into account, for example, that a large number of sources can instill confidence in the data's reliability, and multi-granularity assessment, to evaluate data quality at various aggregation levels [10].…”

Section: G Scalia Et Al / Towards a Scientific Data Framework To Sumentioning

confidence: 99%

Towards a scientific data framework to support scientific model development

Scalia

Pelucchi

Stagni

et al. 2019

View full text Add to dashboard Cite

Editor: Silvio Peroni (https://orcid.org/0000-0003-0530-4305) Solicited reviews: HannaĆwiek-Kupczyńska (https://orcid.org/0000-0001-9113-567X); Aliaksandr Birukou (https://orcid.org/0000-0002-4925-9131); Stian Soiland-Reyes (https://orcid.org/0000-0001-9842-9718)Abstract. The sharing of scientific and scholarly data has been increasingly promoted over the last decade, leading to open repositories in many different scientific domains. However, data sharing and open data are not final goals in themselves, the real benefit is in data reuse, which allows leveraging investments in research and enables large-scale data-driven research progress.This article is published online with Open Access and distributed under the terms of the Creative Commons Attribution License (CC BY 4.0). 2451-8484 © 2019 -IOS Press and the authors. G. Scalia et al. / Towards a scientific data framework to support scientific model developmentFocusing on reuse, this paper discusses the design of an integrated framework to automatically take advantage of large amounts of scientific data extracted from the literature to support research, and in particular scientific model development. Scientific models reproduce and predict complex phenomena and their development is a rather challenging task, within which scientific experiments have a key role in their continuous validation. Starting from the combustion kinetics domain, this paper discusses a set of use cases and a first prototype for such a framework which leads to a set of new requirements and an architecture that can be generalized to other domains. The paper analyzes the needs, the challenges and the research directions for such a framework, in particular those related to data management, automatic scientific model validation, data aggregation and data analysis, to leverage large amounts of published scientific data for new knowledge extraction.

show abstract

Context-aware data quality assessment for big data

Cited by 93 publications

References 22 publications

Big Data Quality Dimensions: A Systematic Literature Review

Big Data Quality Dimensions: A Systematic Literature Review

Methodological Assessment of Data Suitability for Defect Prediction

Towards a scientific data framework to support scientific model development

Contact Info

Product

Resources

About