Evaluating machine learning for information extraction

Ireson, Neil; Ciravegna, Fabio; Califf, Mary Elaine; Freitag, Dayne; Kushmerick, Nicholas; Lavelli, Alberto

doi:10.1145/1102351.1102395

Cited by 33 publications

(20 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The highest values are 10, very rarely 15 [7,8]. Note that a window size of n means to take into account information about 2n + 1 words.…”

Section: Windowingmentioning

confidence: 99%

“…Support Vector Machines have shown to be very suitable for information extraction and are widely used in recent systems [5,14,15,8,7,9,10]. A number of advantages qualify them in order to be used for information extraction:…”

Section: Selection Of the Classification Algorithmmentioning

confidence: 99%

“…The SoftMealy system induces finite state transducers using XML tags as states [6]. 7 http://www.w3.org/TR/xpath…”

Section: Usage Of Structure Informationmentioning

confidence: 99%

See 2 more Smart Citations

Segmentation of legal documents

Mencía

2009

Proceedings of the 12th International Conference on Artificial Intelligence and Law

View full text Add to dashboard Cite

An overwhelming number of legal documents is available in digital form. However, most of the texts are usually only provided in a semi-structured form, i.e. the documents are structured only implicitly using text formatting and alignment. In this form the documents are perfectly understandable by a human, but not by a machine. This is an obstacle towards advanced intelligent legal information retrieval and knowledge systems. The reason for this lack of structured knowledge is that the conversion of texts in conventional form into a structured, machine-readable form, a process called segmentation, is frequently done manually and is therefore very expensive.We introduce a trainable system based on state-of-the-art Information Extraction techniques for the automatic segmentation of legal documents. Our system makes special use of the implicitly given structure in the source digital file as well as of the explicit knowledge about the target structure. Our evaluation on the French IPR Law demonstrates that the system is able to learn an effective segmenter given only a few manually processed training documents. In some cases, even only one seen example is sufficient in order to correctly process the remaining documents.

show abstract

“…The highest values are 10, very rarely 15 [7,8]. Note that a window size of n means to take into account information about 2n + 1 words.…”

Section: Windowingmentioning

confidence: 99%

Section: Selection Of the Classification Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Segmentation of legal documents

Mencía

2009

Proceedings of the 12th International Conference on Artificial Intelligence and Law

View full text Add to dashboard Cite

show abstract

“…The only compulsory task is task1, which used 400 annotated documents for training and other 200 annotated documents for testing. See Ireson and Ciravegna (2005) for a short overview of the challenge. The learning methods explored by the participating systems included LP 2 , HMM, CRF, SVM, and a variety of combinations Table 2: Comparison to other systems on the jobs corpus: F 1 (%) on each entity type and overall performance as macro-averaged F 1 .…”

Section: Comparison To Other Systemsmentioning

confidence: 99%

“…of different learning algorithms. Firstly, the system of the challenge organisers, which is based on LP 2 obtained the best result for Task1, followed by one of our participating systems which combined the uneven margins SVM and PAUM (see Ireson and Ciravegna (2005)). Our SVM and PAUM systems on their own were respectively in the fourth and fifth position among the 20 participating systems.…”

Section: Comparison To Other Systemsmentioning

confidence: 99%

Using uneven margins SVM and perceptron for information extraction

Li¹,

Bontcheva²,

Cunningham³

2005

Proceedings of the Ninth Conference on Computational Natural Language Learning - CONLL '05

View full text Add to dashboard Cite

The classification problem derived from information extraction (IE) has an imbalanced training set. This is particularly true when learning from smaller datasets which often have a few positive training examples and many negative ones. This paper takes two popular IE algorithms -SVM and Perceptron -and demonstrates how the introduction of an uneven margins parameter can improve the results on imbalanced training data in IE. Our experiments demonstrate that the uneven margin was indeed helpful, especially when learning from few examples. Essentially, the smaller the training set is, the more beneficial the uneven margin can be. We also compare our systems to other state-of-theart algorithms on several benchmarking corpora for IE.

show abstract

Requirements and use of in‐service information in an engineering redesign task: Case studies from the aerospace industry

Jagtap

Johnson

2010

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

This paper describes the research stimulated by a fundamental shift that is occurring in the manufacture and marketing of aero engines for commercial and defence purposes, away from the selling of products to the provision of services. This research was undertaken in an aerospace company, which designs and manufactures aero engines and also offers contracts, under which it remains responsible for the maintenance of engines. These contracts allow the company to collect far more data about the in-service performance of their engines than was previously available. This paper aims at identifying what parts of this in-service information are required when components or systems of existing engines need to be redesigned, because they have not performed as expected in service. In addition, this paper aims at understanding how designers use this in-service information in a redesign task. In an attempt to address these aims we analysed five case studies involving redesign of components or systems of an existing engine. The findings show that the in-service information accessed by the designers mainly contains the undesired physical actions (e.g. deterioration mechanisms, deterioration effects, etc.) and the causal chains of these undesired physical actions. We identified a pattern in the designers' actions regarding the use of these causal chains. The designers have generated several solutions that utilize these causal chains seen in the in-service information. The findings provide a sound basis for developing tools and methods to support designers in satisfying their in-service information requirements effectively in a redesign task.

show abstract

Evaluating machine learning for information extraction

Cited by 33 publications

References 3 publications

Segmentation of legal documents

Segmentation of legal documents

Using uneven margins SVM and perceptron for information extraction

Requirements and use of in‐service information in an engineering redesign task: Case studies from the aerospace industry

Contact Info

Product

Resources

About