Crowdsourcing Scholarly Discourse Annotations

Oelen, Allard; Stocker, Markus; Auer, Sören

doi:10.1145/3397481.3450685

Cited by 12 publications

(12 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar PDF annotation tool exists for the ORKG as well and is described by Oelen et al [62]. While this approach seems viable for already published papers, I would argue that adding the annotations directly in L A T E X instead of first converting to PDF is a better approach since it is simpler and less error-prone.…”

Section: Crowdsourced Semantic Annotationmentioning

confidence: 99%

“…Five commands were reserved for the most important properties describing a scientific contribution: research problem, background, method, result and conclusion. These command names were chosen from the DEO classes (see section 3.2) as suggested by the approach of Oelen et al [62]. The DEO class of research statement was adapted to the research problem property which is a central concept of the ORKG vocabulary.…”

Section: Mark Upmentioning

confidence: 99%

See 1 more Smart Citation

SciKGTeX - A LATEX Package to Semantically Annotate Contributions in Scientific Publications

Bless,

Baimuratov,

Karras

2023

2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

View full text Add to dashboard Cite

The continuously increasing output of published research makes the work of researchers harder as it becomes impossible to keep track of and compare the most recent advances in a field. Scientific knowledge graphs have been proposed as a solution to structure the content of research publications in a machine-readable way and enable more efficient, computer-assisted workflows for many research activities. Crowdsourcing approaches are used frequently to build and maintain such scientific knowledge graphs. Researchers are motivated to contribute to these crowdsourcing efforts as they want their work to be included in the knowledge graphs and benefit from applications built on top of them. To contribute to scientific knowledge graphs, researchers need simple and easy-to-use solutions to generate new knowledge graph elements and establish the practice of semantic representations in scientific communication.In this thesis, I present SciKGTeX, a L A T E X package to semantically annotate scientific contributions at the time of document creation. The L A T E X package allows authors of scientific publications to mark the main contributions such as the background, research problem, method, results and conclusion of their work directly in L A T E X source files. The package then automatically embeds them as metadata into the generated PDF document. In addition to the package, I document a user evaluation with 26 participants which I conducted to assess the usability and feasibility of the solution.The analysis of the evaluation results shows that SciKGTeX is highly usable with a score of 79 out of 100 on the System Usability Scale. Furthermore, the study showed that the functionalities of the package can be picked up very quickly by the study participants which only needed 7 minutes on average to annotate the main contributions on a sample abstract of a published paper. SciKGTeX demonstrates a new way to generate structured metadata for the key contributions of research publications and embed them into PDF files at the time of document creation. nische Informationsbibliothek (TIB) in Hannover for allowing me to work together with his group in the context of the ORKG project. Many of the colleagues at TIB have given me great feedback on SciKGTeX and helped me to achieve my evaluation goals. I hope the outcome of this work will ultimately contribute to their vision of an open scientific knowledge graph.Lastly, I thank all the anonymous participants in the user evaluation of this work for their time and feedback. A specific thanks goes to Dr. Umutcan Şimşek for letting me present my thesis in his Seminar and gaining valuable insights as well as a few more evaluation participants.

show abstract

Section: Crowdsourced Semantic Annotationmentioning

confidence: 99%

Section: Mark Upmentioning

confidence: 99%

SciKGTeX - A LATEX Package to Semantically Annotate Contributions in Scientific Publications

Bless,

Baimuratov,

Karras

2023

2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

View full text Add to dashboard Cite

show abstract

“…Figure 1 shows an example of an abstract with classified sentences. Such a semantification of sentences can help to focus on relevant elements of text and thus assist information retrieval systems [50,60] or knowledge graph population [51]. The task is called sequential to distinguish it from the general sentence classification task where a sentence is classified in isolation, i.e.…”

Section: Introductionmentioning

confidence: 99%

Cross-domain multi-task learning for sequential sentence classification in research papers

Brack

Hoppe

Buschermöhle

et al. 2022

Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries

View full text Add to dashboard Cite

Sequential sentence classification deals with the categorisation of sentences based on their content and context. Applied to scientific texts, it enables the automatic structuring of research papers and the improvement of academic search engines. However, previous work has not investigated the potential of transfer learning for sentence classification across different scientific domains and the issue of different text structure of full papers and abstracts. In this paper, we derive seven related research questions and present several contributions to address them: First, we suggest a novel uniform deep learning architecture and multi-task learning for cross-domain sequential sentence classification in scientific texts. Second, we tailor two common transfer learning methods, sequential transfer learning and multi-task learning, to deal with the challenges of the given task. Semantic relatedness of tasks is a prerequisite for successful transfer learning of neural models. Consequently, our third contribution is an approach to semi-automatically identify semantically related classes from different annotation schemes and we present an analysis of four annotation schemes. Comprehensive experimental results indicate that models, which are trained on datasets from different scientific domains, benefit from one another when using the proposed multi-task learning architecture. We also report comparisons with several state-of-the-art approaches. Our approach outperforms the state of the art on full paper datasets significantly while being on par for datasets consisting of abstracts.

show abstract

“…TinyGenius is specifically designed to be integrated into the Open Research Knowledge Graph (ORKG) [7]. The ORKG leverages a crowdsourcing approach to curate a scholarly knowledge graph [14].…”

Section: Introductionmentioning

confidence: 99%

TinyGenius: Intertwining Natural Language Processing with Microtask Crowdsourcing for Scholarly Knowledge Graph Creation

Oelen,

Stocker,

Auer

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Crowdsourcing Scholarly Discourse Annotations

Cited by 12 publications

References 51 publications

SciKGTeX - A LATEX Package to Semantically Annotate Contributions in Scientific Publications

SciKGTeX - A LATEX Package to Semantically Annotate Contributions in Scientific Publications

Cross-domain multi-task learning for sequential sentence classification in research papers

TinyGenius: Intertwining Natural Language Processing with Microtask Crowdsourcing for Scholarly Knowledge Graph Creation

Contact Info

Product

Resources

About