A Survey of Data Quality Measurement and Monitoring Tools

Abstract

High-quality data is key to interpretable and trustworthy data analytics and the basis for meaningful data-driven decisions. In practical scenarios, data quality is typically associated with data preprocessing, profiling, and cleansing for subsequent tasks like data integration or data analytics. However, from a scientific perspective, a lot of research has been published about the measurement (i.e., the detection) of data quality issues and different generally applicable data quality dimensions and metrics have been discussed. In this work, we close the gap between data quality research and practical implementations with a detailed investigation on how data quality measurement and monitoring concepts are implemented in state-of-the-art tools. For the first time and in contrast to all existing data quality tool surveys, we conducted a systematic search, in which we identified 667 software tools dedicated to “data quality.” To evaluate the tools, we compiled a requirements catalog with three functionality areas: (1) data profiling, (2) data quality measurement in terms of metrics, and (3) automated data quality monitoring. Using a set of predefined exclusion criteria, we selected 13 tools (8 commercial and 5 open-source tools) that provide the investigated features and are not limited to a specific domain for detailed investigation. On the one hand, this survey allows a critical discussion of concepts that are widely accepted in research, but hardly implemented in any tool observed, for example, generally applicable data quality metrics. On the other hand, it reveals potential for functional enhancement of data quality tools and supports practitioners in the selection of appropriate tools for a given use case.

A visual analytics approach for the assessment of information quality of performance models—a software review

“…Analyzing and steering Data Quality in analytical processes are very relevant activities in computer science and data analysis [e.g., Ahmed et al (2018) for improvement of data quality in intelligent e-CRM applications, or Vielberth et al (2021) for security incidents], where many automatic tools exist that support these tasks, be it to different degrees. Recent work by Ehrlinger et al (2019) surveyed 667 software tools dedicated to data quality. Among other considerations, the authors report on the large heterogeneity of the tools, evidencing several limitations: (1) more than half of them work only with proprietary solutions; (2) most of them lack implementation of important Data Quality dimensions identified in the state-of-the-art literature, (3) most of them do not support comparability due to the way in which metrics are defined, and (4) they lack user interaction and exploration capabilities supported by visualization for data quality analysis (most of them only focus on usability of basic GUIs useful just to conduct the analysis and not to explore results or support hypothesis forming and testing).…”

On the Evaluation, Management and Improvement of Data Quality in Streaming Time Series

“…This work has been motivated by the identification of the need to monitor quality metrics in time series to ensure the high DQ over time through the possibility of correcting the problems identified [37]. In addition, the available tools do not finish solving the implementation of quality metrics.…”

Ferramentas open-source de qualidade de dados para licitações públicas: Uma análise comparativa

“…Ferramenta dedicada à preparac ¸ão e qualidade de dados. A ferramenta AP fornece uma plataforma integrada de gerenciamento de dados que, além de recursos voltados para a preparac ¸ão de dados, também fornece limpeza de dados, análises estatísticas, correspondência de padrões e a criac ¸ão de profiling de dados [Ehrlinger and Wöß 2022].…”

“…Ferramenta voltada para big data, dedicada a medir continuamente a qualidade de dados em lote ou em streaming. A ferramenta AG oferece um conjunto de modelos de domínio de qualidade de dados bem definidos, que cobre diferentes problemas de qualidade de dados em geral [Ehrlinger and Wöß 2022].…”

“…A ferramenta foi inspirada em um projeto interno da Ubisoft Entertainment para medir e melhorar a qualidade dos dados de sua Enterprise Data Platform. No entanto, a versão open-source foi reformulada para melhorar seu design e remover dependências técnicas com software comercial [Ehrlinger and Wöß 2022].…”