SANTO: A Web-based Annotation Tool for Ontology-driven Slot Filling

Hartung, Matthias; Horst, Hendrik ter; Grimm, Frank; Diekmann, Tim; Klinger, Roman; Cimiano, Philipp

doi:10.18653/v1/p18-4012

Cited by 7 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, some of the tools evaluated here were released in 2018 or 2019, e.g. APLenty [ 32 ], KCAT [ 55 ], PDFAnno [ 85 ], SANTO [ 65 ], SLATE [ 69 ], WARP-Text [ 64 ] and WASA [ 74 ]. Further, by observing criteria P1 (year of the last publication) and T1 (date of the last version), we conclude that nine of the selected tools were published since 2009, and eight tools released their latest version (or source code’s commit) since 2014.…”

Section: Discussionmentioning

confidence: 99%

An extensive review of tools for manual annotation of documents

Neves

Ševa

2019

Briefings in Bioinformatics

View full text Add to dashboard Cite

Motivation Annotation tools are applied to build training and test corpora, which are essential for the development and evaluation of new natural language processing algorithms. Further, annotation tools are also used to extract new information for a particular use case. However, owing to the high number of existing annotation tools, finding the one that best fits particular needs is a demanding task that requires searching the scientific literature followed by installing and trying various tools. Methods We searched for annotation tools and selected a subset of them according to five requirements with which they should comply, such as being Web-based or supporting the definition of a schema. We installed the selected tools (when necessary), carried out hands-on experiments and evaluated them using 26 criteria that covered functional and technical aspects. We defined each criterion on three levels of matches and a score for the final evaluation of the tools. Results We evaluated 78 tools and selected the following 15 for a detailed evaluation: BioQRator, brat, Catma, Djangology, ezTag, FLAT, LightTag, MAT, MyMiner, PDFAnno, prodigy, tagtog, TextAE, WAT-SL and WebAnno. Full compliance with our 26 criteria ranged from only 9 up to 20 criteria, which demonstrated that some tools are comprehensive and mature enough to be used on most annotation projects. The highest score of 0.81 was obtained by WebAnno (of a maximum value of 1.0).

show abstract

Section: Discussionmentioning

confidence: 99%

An extensive review of tools for manual annotation of documents

Neves

Ševa

2019

Briefings in Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Yedda (Yang et al, 2018) offers label recommendations via machine learning and provides both command line and web-based interfaces. SANTO (Hartung et al, 2018), which is designed primarily for slot-filling tasks, enables the formation of relational structures from an ontology. It also visualises the annotations of every user at once to help project owners monitor and curate the quality of the annotations.…”

Section: Related Workmentioning

confidence: 99%

Redcoat: A Collaborative Annotation Tool for Hierarchical Entity Typing

Stewart¹,

Liu²,

Cardell-Oliver³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

We introduce Redcoat, a web-based annotation tool that supports collaborative hierarchical entity typing. As an annotation tool, Redcoat also facilitates knowledge elicitation by allowing the creation and continuous refinement of concept hierarchies during annotation. It aims to minimise not only annotation time but the time it takes for project creators to set up and distribute projects to annotators. Projects created using the web-based interface can be rapidly distributed to a list of email addresses. Redcoat handles the propagation of documents amongst annotators and automatically scales the annotation workload depending on the number of active annotators. In this paper we discuss these key features and outline Redcoat's system architecture. We also highlight Redcoat's unique benefits over existing annotation tools via a qualitative comparison.

show abstract

“…Following an ontology-based approach [26], we assume that these templates (including slots and types of their potential fillers) are pre-defined in a given ontology. 4 Consider the following input document:…”

Section: Introductionmentioning

confidence: 99%

“…This tutorial ends with Section 5 in which we conclude our proposed approach. Parts of the materials presented here are taken from our previous publications [4,7,8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cold-Start Knowledge Base Population Using Ontology-Based Information Extraction with Conditional Random Fields

Horst

Hartung

Cimiano

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In this tutorial we discuss how Conditional Random Fields can be applied to knowledge base population tasks. We are in particular interested in the cold-start setting which assumes as given an ontology that models classes and properties relevant for the domain of interest, and an empty knowledge base that needs to be populated from unstructured text. More specifically, cold-start knowledge base population consists in predicting semantic structures from an input document that instantiate classes and properties as defined in the ontology. Considering knowledge base population as structure prediction, we frame the task as a statistical inference problem which aims at predicting the most likely assignment to a set of ontologically grounded output variables given an input document. In order to model the conditional distribution of these output variables given the input variables derived from the text, we follow the approach adopted in Conditional Random Fields. We decompose the cold-start knowledge base population task into the specific problems of entity recognition, entity linking and slot-filling, and show how they can be modeled using Conditional Random Fields.

show abstract

SANTO: A Web-based Annotation Tool for Ontology-driven Slot Filling

Cited by 7 publications

References 13 publications

An extensive review of tools for manual annotation of documents

An extensive review of tools for manual annotation of documents

Redcoat: A Collaborative Annotation Tool for Hierarchical Entity Typing

Cold-Start Knowledge Base Population Using Ontology-Based Information Extraction with Conditional Random Fields

Contact Info

Product

Resources

About