InfoXtract: A customizable intermediate level information extraction engine

Abstract

Information Extraction (IE) systems assist analysts to assimilate information from electronic documents. This paper focuses on IE tasks designed to support information discovery applications. Since information discovery implies examining large volumes of heterogeneous documents for situations that cannot be anticipated a priori, they require IE systems to have breadth as well as depth. This implies the need for a domain-independent IE system that can easily be customized for specific domains: end users must be given tools to customize the system on their own. It also implies the need for defining new intermediate level IE tasks that are richer than the subject-verb-object (SVO) triples produced by shallow systems, yet not as complex as the domain-specific scenarios defined by the Message Understanding Conference (MUC). This paper describes InfoXtract, a robust, scalable, intermediate-level IE engine that can be ported to various domains. It describes new IE tasks such as synthesis of entity profiles, and extraction of concept-based general events which represent realistic near-term goals focused on deriving useful, actionable information. Entity profiles consolidate information about a person/organization/location etc. within a document and across documents into a single template; this takes into account aliases and anaphoric references as well as key relationships and events pertaining to that entity. Concept-based events attempt to normalize information such as time expressions (e.g., yesterday) as well as ambiguous location references (e.g., Buffalo). These new tasks facilitate the correlation of output from an IE engine with structured data to enable text mining. InfoXtract's hybrid architecture comprised of grammatical processing and machine learning is described in detail. Benchmarking results for the core engine and applications utilizing the engine are presented.

An Information-Extraction System for Urdu---A Resource-Poor Language

“…These are considered Location Entities). Srihari et al [2008] discuss the architecture and the grammar framework of Semantex TM in detail. We have used the grammar toolkit of Semantex TM to develop the morphological analyzer for Urdu.…”

Ontology‐supported polarity mining

“…Further, text decomposition in accordance with semantics-based text themes (Salton, Singhal, Buckley, & Mitra, 1996) is an important future task. In particular, information extraction (Srihari, Li, Cornel, & Niu, 2006), feature identification (Popescu & Etzioni, 2005), aspect identification (Kobayashi, Iida, Inui, & Matsumoto, 2006), or discourse analysis (Teufel & Moens, 2002) facilitate the identification of text segments related to specific properties of an ontology concept. For example, 13 sentential features were extracted to classify scientific articles into a fixed set of rhetorical categories (Teufel & Moens, 2002).…”

Improving Knowledge Discovery in Document Collections through Combining Text Retrieval and Link Analysis Techniques

“…An Information Extraction (IE) engine, Semantex [2] was used to extract the features from document collections. Semantex tags named entities, common relationships associated with person and organization, as well as providing subject-verb-object (SVO) relationships.…”

“…Formally, given the source topic c 1 and destination topic c n , a concept chain CC 1->n is basically a sequence of concepts c 1 -> c 2 -> c 3 , …., c n-1 ->c n , the transitive strength of a path from c 1 to c n made up of the links { (c 1 , c 2 ),…, (c n-1 , c n )}, denoted by g(c 1 , c n ), is given by the following equation: (2) where ω(c i , c i+1 ) is the weight of the edge between c i and c i+1 in the CAG. We call a path an optimal path from c 1 to c n with path length l if its transitive strength is maximal among all the possible paths with length l.…”

“…The narratives tables of this dataset including the aviation accident reports coming from the year 2001 to 2003 were used. All the documents were preprocessed by the Semantex information extraction engine [2], and the concepts are extracted and mapped to the domain ontology as described in section 4. A thesaurus file is used for matching synonyms and acronyms in the texts.…”