Estimating data accuracy in a federated database environment

Abstract

Abstract:The need for integration of data in a heterogeneous or federated database environment creates a corresponding need for estimating the accuracy of the integrated data as a function of the accuracy of the originating data sources. Even in a single database system, different base relations are frequently characterized by dissimilar levels of accuracy; however, no technique exists for defining the accuracy of this single database system in terms of the accuracy of the base relations. This need is further heightened in the case of federated environments involving multiple heterogeneous databases. To address this need, a generalized method is proposed for estimating the overall data accuracy in terms of the accuracy of relevant base relations and the actual database query. The query is examined in terms of its underlying set of base operators. A rigorous theoretical framework encompassing all these possible base operators is presented in this paper using the relational model. While the accuracy estimates are postulated on the basis of uniform distribution, the implications of nonuniform error distributions are also examined in theoretical terms. Finally, a running example is utilized to highlight the practical implications of the proposed theoretical framework.

Assessing Data Quality for Information Products: Impact of Selection, Projection, and Cartesian Product

“…First, users are often provided information in tabular form. Second, the more detailed the granularity, the more expensive it is to measure and represent the quality metrics (Reddy and Wang 1995). Our methodology, however, is general, and we show how it can be applied in situations where quality metrics are available at the attribute level.…”

“…They do not, however, provide a methodology for deriving quality metrics for the output. Reddy and Wang (1995) provide an analysis of the error propagation process when only inaccuracies and mismembers are present. Incompleteness is a critical data quality attribute, in particular for data warehousing applications that draw upon multiple internal and external data sources.…”

Not All Answers are Equally Good: Estimating the Quality of Database Answers

“…It is easy to verify that soundness and completeness satisfy all the requirements of a goodness measure. † Soundness and completeness are similar to precision and recall in information retrieval [Salton, McGill, 1983].…”

FROM QoD TO QoS - Data Quality Issues in Cloud Computing

“…The concept and importance of quality of data has been discussed many times in the literature (Ballou and Pazer, 1985;Batini and Scannapieco, 2006;Tupek, 2006;Wang and Strong, 1996) usually in context of the single data source. However, some research has been also done in the context of integrated data emphasizing the importance of data quality assurance in this context (Gertz and Schmitt, 1998;Naumann, 2002;Reddy and Wang, 1995). Data quality has been also considered in the context of data mining (Berti-Equille, 2007;Dasu and Johnson, 2003).…”