An Algebra for Structured Text Search and a Framework for its Implementation

Abstract

A query algebra is presented that expresses searches on structured text. In addition to traditional fulltext boolean queries that search a pre-defined collection of documents, the algebra permits queries that harness document structure. The algebra manipulates arbitrary intervals of text, which are recognized in the text from implicit or explicit markup. The algebra has seven operators, which combined intervals to yield new ones: containing, not containing, contained in, not contained in, one of, both of, followed by. The ultimate result of a query is the set of intervals that satisfy it. An implementation framework is given based on four primitive access functions. Each access function finds the solution to a query nearest to a given position in the database. Recursive definitions for the seven operators are given in terms of these access functions. Search time is at worst proportional to the time required to evaluate the access functions for occurrences of the elementary terms in a query.

Information Retrieval Models

“…Regions models (Burkowski 1992;Clarke et al 1995;Navarro and Baeza-Yates 1997;Jaakkola and Kilpelainen 1999) Figure 3 shows a fragment from Shakespeare's Hamlet for which we numbered the word positions. The figure shows the region that starts at word 103 and ends at word 131.…”

“…As above, the query <SPEECH> CONTAINING Hamlet retrieves all speeches that contain the word 'Hamlet'. In later publications Clarke et al (1995) and Jaakkola and Kilpelainen (1999) describe region models that do not distinguish mark-up from content. In their system, the operator FOLLOWED BY is needed to match opening and closing tags, so the query would be somewhat more verbose: (<speech> FOLLOWED BY </speech>) CONTAINING Hamlet In some region models, such as the model by Clarke et al (1995) the query A AND B does not retrieve the intersection of sets A and B, but instead retrieves the smallest regions that contain a region from both set A and set B.…”

Tag-Annotated Text Search Using Extended Region Algebra

“…A region expressed by the tags can be any meaningful units such as title, section, or sentence. Region Algebra by [8] defined a set of algebraic operators on the sets of regions. Operators in the original algebra are shown in Table 1 Let us denote the set of regions of the start tag <A>, and one of the end tag </A>, by S(<A>), and S(</A>), respectively.…”

“…The major attraction of the framework of region algebra by [8] is in its efficient algorithms for finding regions one by one that satisfy a query formulated in an algebraic formula. The algorithms for Containing( ) and Followedby ( ) are shown in Fig.…”

“…On the contrary, the functionality of full-text search engines with tags such as that of Lucene [7], is not expressive enough. In this paper, we propose a new retrieval system based on a region algebra [8], and show how operations of the region algebra can handle linguistic annotations effectively and efficiently. We extend the original region algebra in order to treat the nested annotation; this is indispensable for dealing with the hierarchical nature of linguistic representation.…”