Concordance Comparison as a Means of Assembling Local Grammars

Abstract

Named Entity Recognition for person names is an important but non-trivial task in information extraction. This article uses a tool that compares the concordances obtained from two local grammars (LG) and highlights the differences. We used the results as an aid to select the best of a set of LGs. By analyzing the comparisons, we observed relationships of inclusion, intersection and disjunction within each pair of LGs, which helped us to assemble those that yielded the best results. This approach was used in a case study on extraction of person names from texts written in Portuguese. We applied the enhanced grammar to the Gold Collection of the Second HAREM. The F-Measure obtained was 76.86, representing a gain of 6 points in relation to the state-of-the-art for Portuguese.

The Optimization of Portuguese Named-Entity Recognition and Classification by Combining Local Grammars and Conditional Random Fields Trained with a Parsed Corpus

“…The association with other information coming from the parsed corpus showed no further increase, thus, no positive synergy has been identified. This observation corroborates with what is usually seen in the literature concerning CRF for NERC task where, usually, only PoS is used as feature [9].…”

“…Pirovani, J. and De Oliveira, E. [9] also used local grammar and PoS annotations associated with CRF models for NERC task, however, no proof of the synergy between these two types of features was demonstrated. Also, the highest F1 measure obtained by the proposed system (for all HAREM categories, not considering second and third levels of the hierarchy) was 60,4%, while our system achieves 75,7% for the "Time" category considering all its types and sub-types.…”

“…A rule-based model for Portuguese NERC has been proposed inside Port4NooJ v3.0 [8], however, the evaluation of this tool has not been provided and the module is not fully available. Pirovani, J. and De Oliveira, E. proposed in their article [9] the association of local grammars with Conditional Random Fields (CRF) probabilistic method based on a training set containing Part-of-Speech (PoS) tags achieving better results than previous works concerning HAREM. In this study, the authors considered only the following HAREM named-entity categories: "Person", "Place", "Organization", "Time" and "Value".…”

“…The IOB format, proposed by Ramshaw and Marcus [11], which allows a more detailed analysis of the boundary identification of the entities, was chosen for our study following the work of Pirovani and De Oliveira [9].…”