Rewrite rules for search database systems

Abstract

The results of a search engine can be improved by consulting auxiliary data. In a search database system, the association between the user query and the auxiliary data is driven by rewrite rules that augment the user query with a set of alternative queries. This paper develops a framework that formalizes the notion of a rewrite program, which is essentially a collection of hedge-rewriting rules. When applied to a search query, the rewrite program produces a set of alternative queries that constitutes a least fixpoint (lfp). The main focus of the paper is on the lfp-convergence of a rewrite program, where a rewrite program is lfp-convergent if the least fixpoint of every search query is finite. Determining whether a given rewrite program is lfp-convergent is undecidable; to accommodate that, the paper proposes a safety condition, and shows that safety guarantees lfp-convergence, and that safety can be decided in polynomial time. The effectiveness of the safety condition in capturing lfp-convergence is illustrated by an application to a rewrite program in an implemented system that is intended for widespread use.

Next generation data analytics at IBM research

“…A comprehensive tooling framework [15] facilitates development and maintenance of extraction rules, with tools for automatic rule production and refinement. SystemT is also used for backend analytics in an enterprise search system driven by a comprehensive, domain adaptable search architecture developed in IBM Research [1,9]. Similarly, for enabling large-scale statistical analysis and predictive modeling, SystemML [11] implements a declarative, high-level language using an R-like syntax extended with constructs that are machine-learning specific.…”

Cleaning inconsistencies in information extraction via prioritized repairs

“…However, it turns out that it is undecidable to determine whether a pgd guarantees acyclicity of the priority relation. The conclusion is that other (stronger) conditions need to be imposed if we want to automatically verify unambiguity of an extraction program (an example might be by using a potential function, e.g., as in [13]). Such conditions are beyond the scope of this paper, and are left as important directions for future work.…”