Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation

Abstract

In many applications, one can obtain descriptions about the same objects or events from a variety of sources. As a result, this will inevitably lead to data or information conflicts. One important problem is to identify the true information (i.e., the truths) among conflicting sources of data. It is intuitive to trust reliable sources more when deriving the truths, but it is usually unknown which one is more reliable a priori. Moreover, each source possesses a variety of properties with different data types. An accurate estimation of source reliability has to be made by modeling multiple properties in a unified model. Existing conflict resolution work either does not conduct source reliability estimation, or models multiple properties separately. In this paper, we propose to resolve conflicts among multiple sources of heterogeneous data types. We model the problem using an optimization framework where truths and source reliability are defined as two sets of unknown variables. The objective is to minimize the overall weighted deviation between the truths and the multi-source observations where each source is weighted by its reliability. Different loss functions can be incorporated into this framework to recognize the characteristics of various data types, and efficient computation approaches are developed. Experiments on real-world weather, stock and flight data as well as simulated multi-source data demonstrate the necessity of jointly modeling different data types in the proposed framework 1 .

A confidence-aware approach for truth discovery on long-tail data

“…They are compared on both continuous and categorical data unless otherwise specified. The baseline methods include some state-of-the-art truth discovery methods: GTM [36], TruthFinder [34], AccuSim [8], Investment [23], 3-Estimates [16], and CRH [18]. More detailed summary of these methods can be found in Section 5.…”

“…Recently, Dong et al model source selection in the truth discovery tasks based on the idea of "gain" and "cost" [11,27]. Li et al aim to minimize the weighted deviation of claims and truths, so an optimization framework is adopted and applied on heterogeneous data, in which different data types can be modeled jointly [18].…”

Spatio-Temporal Behavior Analysis and Pheromone-Based Fusion Model for Big Trace Data

“…The length of visits, visiting rate, stay points, online searches, check-in data, and photos on social media platforms are attributes analyzed and included in the model based on the pheromone and digital trail analogy. This work provides a new perspective, inspired by research on the behavior of real ants and data fusion [34][35][36][37][38].…”

“…Using a non-dimensional method, the two pheromones in online and offline space are integrated to calculate the aggregate pheromones (M i ) in urban recreational areas. The mathematical Equation (6) shows the normalization, and Equation (7) is the integration with a polynomial expression, inspired by Li et al [37]:…”

Approximate Truth Discovery via Problem Scale Reduction

“…If true, the process diverges into two branches: i) adding the value to the existing group (lines [11][12][13][14] and ii) setting up a new group for the value (lines [15][16][17][18]. In the first case, Algorithm 2 invokes itself, taking the updated solution (with the new value added to the last group) and the updated sublist (with the first value removed) as inputs (line 12).…”