A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools

Verspoor, Karin; Cohen, K. Bretonnel; Lanfranchi, Arrick; Warner, Colin; Johnson, H. L.; Roeder, Christophe; Choi, Jinho D.; Funk, Christopher; Malenkiy, Yuriy; Eckert, Miriam; Xue, Nianwen; Baumgartner, William A.; Bada, Michael; Palmer, Martha; Hunter, Lawrence

doi:10.1186/1471-2105-13-207

Cited by 108 publications

(105 citation statements)

References 45 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…A team consisting of the author, a biology doctoral student with research experience in genetics, and a computer science graduate student with an undergraduate degree in philosophy collaborated on identifying the arguments in the Results section of two articles ( Van de Leemput et al, 2007;Lloyd et al, 2005) from the CRAFT corpus (Bada et al, 2012;Verspoor et al 2012). The CRAFT corpus is open access and has already been linguistically annotated.…”

Section: Preparationmentioning

confidence: 99%

Manual Identification of Arguments with Implicit Conclusions Using Semantic Rules for Argument Mining

Green¹

2017

Proceedings of the 4th Workshop on Argument Mining

View full text Add to dashboard Cite

This paper describes a pilot study to evaluate human analysts' ability to identify the argumentation scheme and premises of an argument having an implicit conclusion. In preparation for the study, argumentation scheme definitions were crafted for genetics research articles. The schemes were defined in semantic terms, following our proposal to use semantic rules to mine arguments in that literature.

show abstract

Section: Preparationmentioning

confidence: 99%

Manual Identification of Arguments with Implicit Conclusions Using Semantic Rules for Argument Mining

Green¹

2017

Proceedings of the 4th Workshop on Argument Mining

View full text Add to dashboard Cite

show abstract

Section: Preparationmentioning

confidence: 99%

Proceedings of the 4th Workshop on Argument Mining

2017

View full text Add to dashboard Cite

ii IntroductionThe goal of this workshop is to provide a follow-on forum to the last three years' Argumentation Mining workshops at ACL and NAACL, the first research forum devoted to argumentation mining in all domains of discourse.Argument mining (also, "argumentation mining", referred to as "computational argumentation" in some recent works) is a relatively new challenge in corpus-based discourse analysis that involves automatically identifying argumentative structures within discourse, e.g., the premises, conclusion, and argumentation scheme of each argument, as well as argument-subargument and argumentcounterargument relationships between pairs of arguments in the document. To date, researchers have investigated methods for argument mining in areas such as legal documents, on-line debates, product reviews, academic literature, user comments on proposed regulations, newspaper articles and court cases, as well as in dialogical domains. To date there are few corpora with annotations for argumentation mining research although corpora with annotations for argument sub-components have recently become available.Proposed applications of argumentation mining include improving information retrieval and information extraction as well as end-user visualization and summarization of arguments. Textual sources of interest include not only the formal writing of legal text, scientific writing and parliamentary records, but also a variety of informal genres such as microtext, spoken meeting transcripts, product reviews and user comments. In instructional contexts where argumentation is a pedagogically important tool for conveying and assessing students' command of course material, the written and diagrammed arguments of students (and the mappings between them) are educational data that can be mined for purposes of assessment and instruction. This is especially important given the wide-spread adoption of computer-supported peer review, computerized essay grading, and large-scale online courses and MOOCs. Success in argument mining will require interdisciplinary approaches informed by natural language processing technology, theories of semantics, pragmatics and discourse, knowledge of discourse of domains such as law and science, artificial intelligence, argumentation theory, and computational models of argumentation. In addition, it will require the creation and annotation of high-quality corpora of argumentation from different types of sources in different domains.We are looking forward to a full day workshop to exchange ideas and present ongoing research on all of the above -see you all in Copenhagen, Denmark at EMNLP 2017! AbstractIn this paper we present the dataset of 200,000+ political arguments produced in the local phase of the 2016 Chilean constitutional process. We describe the human processing of this data by the government officials, and the manual tagging of arguments performed by members of our research group. Afterwards we focus on classification tasks that mimic the human processes, comparing linear methods ...

show abstract

“…The Yang et al (2015) system uses LingPipe and ClearNLP 7 (Choi and Palmer, 2011) to parse the questions and relevant snippets using models applicable to generic English texts as well as biomedical texts, e.g. the parser models trained on the CRAFT treebank (Verspoor et al, 2012). It uses the named entity recognition (NER) module from LingPipe 8 trained on the GENIA corpus (Kim et al, 2003) and MetaMap annotation component (Aronson, 2001) to identify the biomedical concepts, and further uses UMLS Terminology Services (UTS) 9 to identify concepts and retrieve synonyms.…”

Section: Overview Of Yang Et Al (2015) Systemmentioning

confidence: 99%

Proceedings of the Fourth BioASQ workshop

2016

View full text Add to dashboard Cite

ISBN 978-1-945626-22-7 ACL anthology W16-31xxii PrefaceThe fourth BioASQ workshop on biomedical semantic indexing and question answering took place in Berlin, Germany on August 13th, 2016 and was hosted by the Humboldt University. The workshop was supported by the BioASQ project 1 , which organizes the corresponding annual challenge. The goals of the workshop were to present the results of the fourth BioASQ challenge and further the interaction with the wider community of biomedical semantic indexing and question answering. The presenters represented research teams from different parts of the globe and with different viewpoints to the problem. This contributed to a very lively and interesting discussion among the participants of the workshop. Six papers were presented during the workshop. All were selected by peer review for presentation. This volume includes 7 papers and one abstract: The first paper gives an overview of the challenge, including especially the datasets that were used throughout the challenges and the overall results achieved by the participants. The remaining six papers are those presented at the workshop. The first of these papers is a new extension of the MTI system. In particular, Learning to Rank methodology is used as a boosting component of the MTI system. The second paper present a system which includes several multi-label classifiers (MLC) that are combined in ensembles. Elastic-Search for indexing is the object of discourse of the third workshop paper. The fourth paper presents a system which uses TmTool in addition to MetaMap, to identify possible biomedical named entities, especially out-of-vocabulary concepts. They also introduced a unified classification interface for judging the relevance of each retrieved concept, document, and snippet, which can combine the relevant scores evidenced by various sources. The system presented in the fifth paper relies on the Hana Database for text processing. It uses the Stanford CoreNLP package for tokenizing the questions. Each of the tokens is then sent to the BioPortal and to the Hana database for concept retrieval. The concepts retrieved from the two stores are finally merged to a single list that is used to retrieve relevant text passages from the documents at hand. The last paper focuses on the retrieval of relevant documents and snippets. The proposed system uses a cluster-based language model. Then, it reranks the retrieved top-n sentences using five independent similarity models based on shallow semantic analysis.Finally, the proceedings also include the abstract of one paper that was presented in the poster session only, which describes an approach for extending the web services in order to retrieve the relevant documents, concepts, snippets and triples for the question-answering task.We wish to thank all who participated to the success of this workshop, especially the authors, reviewers, speakers and participants. AbstractThe goal of this task is to push the research frontier towards hybrid information systems. We aim to promote syste...

show abstract

A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools

Cited by 108 publications

References 45 publications

Manual Identification of Arguments with Implicit Conclusions Using Semantic Rules for Argument Mining

Manual Identification of Arguments with Implicit Conclusions Using Semantic Rules for Argument Mining

Proceedings of the 4th Workshop on Argument Mining

Proceedings of the Fourth BioASQ workshop

Contact Info

Product

Resources

About