Managing Uncertainty: The Road Towards Better Data Interoperability

Abstract

Data interoperability encompasses the many data management activities needed for effective information management in anyone´s or any organization´s everyday work such as data cleaning, coupling, fusion, mapping, and information extraction. It is our conviction that a significant amount of money and time in IT that is devoted to these activities, is about dealing with one problem: “semantic uncertainty”. Sometimes data is subjective, incomplete, not current, or incorrect, sometimes it can be interpreted in different ways, etc. In our opinion, clean correct data is only a special case, hence data management technology should treat data quality problems as a fact of life, not as something to be repaired afterwards. Recent approaches treat uncertainty as an additional source of information which should be preserved to reduce its impact. We believe that the road towards better data interoperability, is to be found in teaching our data processing tools and systems about all forms of doubt and how to live with them. In this paper, we show for several data interoperability use cases (deduplication, data coupling/fusion, and information extraction) how to formally model the associated data quality problems as semantic uncertainty. Furthermore, we provide an argument why our approach leads to better data interoperability in terms of natural problem exposure and risk assessment, more robustness and automation, reduced development costs, and potential for natural and effective feedback loops leveraging human attention.

Sample-based XPath Ranking for Web Information Extraction

“…Generating more XPaths may improve the ability to find the most suitable XPath. Also, the use of a probabilistic database approach may be able to more robustly address ambiguous situations [17,18].…”

Uncertain Groupings: Probabilistic Combination of Grouping Data

“…The formalization is based on [13] which provides a generic formalization of a probabilistic database. We summarize the main concepts of [13] (Definitions) and show how it can be specialized to support uncertain groupings (Specializations). Table 1 gives an overview of our notation.…”

“…We like to emphasize that the above is a summary of the main concepts of [13] which provides a generic formalization of a probabilistic database. In addition to summarizing, we have also shown how the formalization can be specialized to support uncertain groupings.…”

“…In addition to summarizing, we have also shown how the formalization can be specialized to support uncertain groupings. For a more detailed presentation of the generic formalization, we refer to [13].…”

Large scale mining and evidence combination to support medical diagnosis

“…This model, as well as several others in the database community such as MayBMS [82,83], is based on possible worlds semantics. An important application domain for probabilistic databases is data integration [84], because the explicit treatment of the uncertainty surrounding (integrated) data is an important technique for achieving better data quality [85].…”