Data Quality Issues Concerning Statistical Data Gathering Supported by Big Data Technology

Maślankowski, Jacek

doi:10.1007/978-3-319-06932-6_10

Cited by 5 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These studies generally elaborated on the issues and proposed generic frameworks with no comprehensive approaches and techniques to manage quality across the Big Data lifecycle. Among these, generic frameworks are presented in [5,39,40].…”

Section: Big Data Lifecycle: Where Quality Matters?mentioning

confidence: 99%

Big data quality framework: a holistic approach to continuous quality management

et al. 2021

View full text Add to dashboard Cite

Big Data is an essential research area for governments, institutions, and private agencies to support their analytics decisions. Big Data refers to all about data, how it is collected, processed, and analyzed to generate value-added data-driven insights and decisions. Degradation in Data Quality may result in unpredictable consequences. In this case, confidence and worthiness in the data and its source are lost. In the Big Data context, data characteristics, such as volume, multi-heterogeneous data sources, and fast data generation, increase the risk of quality degradation and require efficient mechanisms to check data worthiness. However, ensuring Big Data Quality (BDQ) is a very costly and time-consuming process, since excessive computing resources are required. Maintaining Quality through the Big Data lifecycle requires quality profiling and verification before its processing decision. A BDQ Management Framework for enhancing the pre-processing activities while strengthening data control is proposed. The proposed framework uses a new concept called Big Data Quality Profile. This concept captures quality outline, requirements, attributes, dimensions, scores, and rules. Using Big Data profiling and sampling components of the framework, a faster and efficient data quality estimation is initiated before and after an intermediate pre-processing phase. The exploratory profiling component of the framework plays an initial role in quality profiling; it uses a set of predefined quality metrics to evaluate important data quality dimensions. It generates quality rules by applying various pre-processing activities and their related functions. These rules mainly aim at the Data Quality Profile and result in quality scores for the selected quality attributes. The framework implementation and dataflow management across various quality management processes have been discussed, further some ongoing work on framework evaluation and deployment to support quality evaluation decisions conclude the paper.

show abstract

Section: Big Data Lifecycle: Where Quality Matters?mentioning

confidence: 99%

Big data quality framework: a holistic approach to continuous quality management

et al. 2021

View full text Add to dashboard Cite

show abstract

“…First is the authenticity of (big) data collecting method, which is rather opportunistic than scientific [17][18][19]. This factor may significantly affect representativeness of data sample being used for analysis.…”

Section: B Literature Reviewmentioning

confidence: 99%

Antecedents of big data quality: An empirical examination in financial service organizations

Haryadi

Hulstijn

Wahyudi

et al. 2016

2016 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Abstract-Big data has been acknowledged for its enormous potential. In contrast to the potential, in a recent survey more than half of financial service organizations reported that big data has not delivered the expected value. One of the main reasons for this is related to data quality. The objective of this research is to identify the antecedents of big data quality in financial institutions. This will help to understand how data quality from big data analysis can be improved. For this, a literature review was performed and data was collected using three case studies, followed by content analysis. The overall findings indicate that there are no fundamentally new data quality issues in big data projects. Nevertheless, the complexity of the issues is higher, which makes it harder to assess and attain data quality in big data projects compared to the traditional projects. Ten antecedents of big data quality were identified encompassing data, technology, people, process and procedure, organization, and external aspects.

show abstract

Multi-language Sentiment Analysis – Lesson Learnt from NLP Case Study

Maślankowski

Majewicz

2022

Information Systems

View full text Add to dashboard Cite

Data Quality Issues Concerning Statistical Data Gathering Supported by Big Data Technology

Cited by 5 publications

References 14 publications

Big data quality framework: a holistic approach to continuous quality management

Big data quality framework: a holistic approach to continuous quality management

Antecedents of big data quality: An empirical examination in financial service organizations

Multi-language Sentiment Analysis – Lesson Learnt from NLP Case Study

Contact Info

Product

Resources

About