Preference-aware integration of temporal data

Alexe, Bogdan; Roth, Mary T.; Tan, Wang-Chiew

doi:10.14778/2735496.2735500

Cited by 8 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then study our algorithms in depth with synthetic data. 2 We measure the effectiveness of both implication and duration discoveries. We also measure the execution time needed by the algorithms.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Temporal rules discovery for web data cleaning

Abedjan¹,

Akcora²,

Ouzzani³

et al. 2015

Proc. VLDB Endow.

View full text Add to dashboard Cite

Declarative rules, such as functional dependencies, are widely used for cleaning data. Several systems take them as input for detecting errors and computing a "clean" version of the data. To support domain experts,in specifying these rules, several tools have been proposed to profile the data and mine rules. However, existing discovery techniques have traditionally ignored the time dimension. Recurrent events, such as persons reported in locations, have a duration in which they are valid, and this duration should be part of the rules or the cleaning process would simply fail.In this work, we study the rule discovery problem for temporal web data. Such a discovery process is challenging because of the nature of web data; extracted facts are (i) sparse over time, (ii) reported with delays, and (iii) often reported with errors over the values because of inaccurate sources or non robust extractors. We handle these challenges with a new discovery approach that is more robust to noise. Our solution uses machine learning methods, such as association measures and outlier detection, for the discovery of the rules, together with an aggressive repair of the data in the mining step itself. Our experimental evaluation over real-world data from Recorded Future, an intelligence company that monitors over 700K Web sources, shows that temporal rules improve the quality of the data with an increase of the average precision in the cleaning process from 0.37 to 0.84, and a 40% relative increase in the average F-measure.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Integration and cleaning with temporal data [2,9,25,27] is also of interest. The related approaches can benefit from our algorithms.…”

Section: Related Workmentioning

confidence: 99%

Temporal rules discovery for web data cleaning

Abedjan¹,

Akcora²,

Ouzzani³

et al. 2015

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…• For [ n], observe that [ 2] is a po-relation with a non-empty order, while any query involving the other operators will have empty order (none of our unary and binary operators turns unordered po-relations into an ordered one, and the [t] constant expression produces an unordered po-relation).…”

Section: Incomparability Of Posra Operatorsmentioning

confidence: 99%

Computing possible and certain answers over order-incomplete data

Amarilli

Deutch

et al. 2019

Theoretical Computer Science

View full text Add to dashboard Cite

This paper studies the complexity of query evaluation for databases whose relations are partially ordered; the problem commonly arises when combining or transforming ordered data from multiple sources. We focus on queries in a useful fragment of SQL, namely positive relational algebra with aggregates, whose bag semantics we extend to the partially ordered setting. Our semantics leads to the study of two main computational problems: the possibility and certainty of query answers. We show that these problems are respectively NP-complete and coNPcomplete, but identify tractable cases depending on the query operators or input partial orders. We further introduce a duplicate elimination operator and study its effect on the complexity results.

show abstract

“…However, they mainly focus on how the joined tuples are ranked and selected, instead of how the tables (and in our case sets of objects) are joined to generate the joined tuples. Alexe et al [6] discussed user-defined preference rules for integrating temporal data, which is orthogonal to our work. To the best of our knowledge, [16] is the only work discussing result set preference for joining relational tables.…”

Section: Related Workmentioning

confidence: 99%

“…shows the maximum/minimum possible values of Algorithm 2: Incremental similarity join.Input: thresholds θ i −1 and θ i where θ i −1 > θ i Output: incremental similarity join result join = (θ i )1 join = (θ i ) ← ∅ 2 let cand(θ i ) be the candidate pairs for [θ i , θ i −1 ) 3 foreach pair (r, s) ∈ cand(θ i ) do 4 while sim min (r, s) < θ i ≤ sim max (r, s) do 5update sim max (r, s) and sim min (r, s)6 if sim max (r, s) < θ i then 7 find θ j : θ j −1 > sim max (r, s) ≥ θ j by binary search 8 add (r, s) into cand(θ j ) (r, s) into join = (θ i ) 11 return join = (θ i ) sim(r , s). Through the scanning, we iteratively update sim max (r , s)…”

mentioning

confidence: 99%

Preference-driven similarity join

Gao

Wang

Pei

et al. 2017

Proceedings of the International Conference on Web Intelligence

View full text Add to dashboard Cite

Similarity join, which can find similar objects (e.g., products, names, addresses) across different sources, is powerful in dealing with variety in big data, especially web data. Threshold-driven similarity join, which has been extensively studied in the past, assumes that a user is able to specify a similarity threshold, and then focuses on how to efficiently return the object pairs whose similarities pass the threshold. We argue that the assumption about a well set similarity threshold may not be valid for two reasons. The optimal thresholds for different similarity join tasks may vary a lot. Moreover, the end-to-end time spent on similarity join is likely to be dominated by a back-and-forth threshold-tuning process.In response, we propose preference-driven similarity join. The key idea is to provide several result set preferences, rather than a range of thresholds, for a user to choose from. Intuitively, a result set preference can be considered as an objective function to capture a user's preference on a similarity join result. Once a preference is chosen, we automatically compute the similarity join result optimizing the preference objective. As the proof of concept, we devise two useful preferences and propose a novel preference-driven similarity join framework coupled with effective optimization techniques. Our approaches are evaluated on four real-world web datasets from a diverse range of application scenarios. The experiments show that preference-driven similarity join can achieve high-quality results without a tedious threshold-tuning process.

show abstract

Preference-aware integration of temporal data

Cited by 8 publications

References 30 publications

Temporal rules discovery for web data cleaning

Temporal rules discovery for web data cleaning

Computing possible and certain answers over order-incomplete data

Preference-driven similarity join

Contact Info

Product

Resources

About