Bureau for Rapid Annotation Tool: collaboration can do more among variance annotations

Abstract

PurposeThe purpose of this study is to solve the problems caused by the growing volumes of pre-annotated literature and variety-oriented annotations, including teamwork, quality control and time effort.Design/methodology/approach An annotation collaboration workbench is developed, which is named as Bureau for Rapid Annotation Tool (Brat). Main functionalities include an enhanced semantic constraint system, Vim-like shortcut keys, an annotation filter and a graph-visualizing annotation browser. With these functionalities, the annotators are encouraged to question their initial mindset, inspect conflicts and gain agreement from their peers.FindingsThe collaborative patterns can indeed be leveraged to structure properly every annotator’s behaviors. The Brat workbench can actually be seen as an experienced-based annotation tool by harnessing collective intelligence. Compared to previous counterparts, about one-third of time can be saved on Xinhuanet military news and patent corpora with the workbench.Originality/valueThe various annotations are very popular in real-world annotation tasks with multiple annotators. Though, it is still under-discussed on variety-oriented annotations. The findings of this study provide the practitioners valuable insight into how to govern annotation projects. In addition, the Brat workbench takes the first step for future research on annotating large-scale text resources.

Guest editorial: Extraction and evaluation of knowledge entities in the age of artificial intelligence

“…To address challenges posed by growing volumes of pre-annotated literature and diverse annotations, such as teamwork, quality control and time management, Wang et al . (2023) developed the Bureau for Rapid Annotation Tool (Brat), an annotation collaboration workbench that includes an enhanced semantic constraint system, Vim-like shortcut keys, an annotation filter and a graph-visualizing annotation browser.…”