Uncertainty management in rule-based information extraction systems

Michelakis, Eirinaios; Krishnamurthy, Rajasekar; Haas, Peter J.; Vaithyanathan, Shivakumar

doi:10.1145/1559845.1559858

Cited by 20 publications

(23 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e.g. data collected from sensor networks [15], information extraction from the web [16,17], data integration [18,19], data cleaning [20][21][22][23][24][25], social networks [26,27], radio frequency identification RFID [7]. Due to various reasons that differ from one application to another, the uncertainty is inherent in such applications.…”

Section: Related Workmentioning

confidence: 99%

Framework for Managing Uncertain Distributed Categorical Data

Benaissa¹,

Yahmi²,

Jamil³

2017

ijacsa

View full text Add to dashboard Cite

Abstract-In recent years, data has become uncertain due to the flourishing advanced technologies that participate continuously and increasingly in producing large amounts of incomplete data. Often, many modern applications where uncertainty occurs are distributed in nature, e.g., distributed sensor networks, information extraction, data integration, social network, etc. Consequently, even though the data uncertainty has been studied in the past for centralized behavior, it is still a challenging issue to manage uncertainty over the data in situ. In this paper, we propose a framework to managing uncertain categorical data over distributed environments that is built upon a hierarchical indexing technique based on inverted index, and a distributed algorithm to efficiently process queries on uncertain data in distributed environment. Leveraging this indexing technique, we address two kinds of queries on the distributed uncertain databases 1) a distributed probabilistic thresholds query, where its answers satisfy the probabilistic threshold requirement; and 2) a distributed top-k-queries, optimizing, the transfer of the tuples from the distributed sources to the coordinator site and the time treatment. Extensive experiments are conducted to verify the effectiveness and efficiency of the proposed method in terms of communication costs and response time.

show abstract

Section: Related Workmentioning

confidence: 99%

Framework for Managing Uncertain Distributed Categorical Data

Benaissa¹,

Yahmi²,

Jamil³

2017

ijacsa

View full text Add to dashboard Cite

show abstract

“…al. [30] look at the combination of the annotation rules adopted by each annotator. As in MEMM, it makes use of a maximal entropy classifier, but with "evidence vectors" being the output of rule-based annotators.…”

Section: Related Workmentioning

confidence: 99%

Aggregating semantic annotators

2013

View full text Add to dashboard Cite

A growing number of resources are available for enriching documents with semantic annotations. While originally focused on a few standard classes of annotations, the ecosystem of annotators is now becoming increasingly diverse. Although annotators often have very different vocabularies, with both high-level and specialist concepts, they also have many semantic interconnections. We will show that both the overlap and the diversity in annotator vocabularies motivate the need for semantic annotation integration: middleware that produces a unified annotation on top of diverse semantic annotators. On the one hand, the diversity of vocabulary allows applications to benefit from the much richer vocabulary available in an integrated vocabulary. On the other hand, we present evidence that the most widely-used annotators on the web suffer from serious accuracy deficiencies: the overlap in vocabularies from individual annotators allows an integrated annotator to boost accuracy by exploiting inter-annotator agreement and disagreement.The integration of semantic annotations leads to new challenges, both compared to usual data integration scenarios and to standard aggregation of machine learning tools. We overview an approach to these challenges that performs ontology-aware aggregation. We introduce an approach that requires no training data, making use of ideas from database repair. We experimentally compare this with a supervised approach, which adapts maximal entropy Markov models to the setting of ontology-based annotations. We further experimentally compare both these approaches with respect to ontologyunaware supervised approaches, and to individual annotators.

show abstract

“…The earlier efforts in declarative IE [1,2,3] lack a unified framework supporting both a declarative interface as well as the state-of-the-art probabilistic IE models. Ways to handle uncertainties in IE have been considered in [14,15]. A probabilistic declarative IE system has been proposed in [4,13], but it only supports the Viterbi algorithm, which is unable to handle complex models that arise naturally from advanced features and relational operators.…”

Section: Related Workmentioning

confidence: 99%

Hybrid in-database inference for declarative information extraction

Wang

Franklin

Garofalakis

et al. 2011

Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

In the database community, work on information extraction (IE) has centered on two themes: how to effectively manage IE tasks, and how to manage the uncertainties that arise in the IE process in a scalable manner. Recent work has proposed a probabilistic database (PDB) based declarative IE system that supports a leading statistical IE model, and an associated inference algorithm to answer top-k-style queries over the probabilistic IE outcome. Still, the broader problem of effectively supporting general probabilistic inference inside a PDB-based declarative IE system remains open. In this paper, we explore the in-database implementations of a wide variety of inference algorithms suited to IE, including two Markov chain Monte Carlo algorithms, Viterbi and sum-product algorithms. We describe the rules for choosing appropriate inference algorithms based on the model, the query and the text, considering the trade-off between accuracy and runtime. Based on these rules, we describe a hybrid approach to optimize the execution of a single probabilistic IE query to employ different inference algorithms appropriate for different records. We show that our techniques can achieve up to 10-fold speedups compared to the non-hybrid solutions proposed in the literature.

show abstract

Uncertainty management in rule-based information extraction systems

Cited by 20 publications

References 31 publications

Framework for Managing Uncertain Distributed Categorical Data

Framework for Managing Uncertain Distributed Categorical Data

Aggregating semantic annotators

Hybrid in-database inference for declarative information extraction

Contact Info

Product

Resources

About