A semi-automatic approach for detecting dataset references in social science texts

Abstract

Abstract. Today, full-texts of scientific articles are often stored in different locations than the used datasets. Dataset registries aim at a closer integration by making datasets citable but authors typically refer to datasets using inconsistent abbreviations and heterogeneous metadata (e.g. title, publication year). It is thus hard to reproduce research results, to access datasets for further analysis, and to determine the impact of a dataset. Manually detecting references to datasets in scientific articles is time-consuming and requires expert knowledge in the underlying research domain. We propose and evaluate a semi-automatic three-step approach for finding explicit references to datasets in social sciences articles. We first extract pre-defined special features from dataset titles in the da|ra registry, then detect references to datasets using the extracted features, and finally match the references found with corresponding dataset titles. The approach does not require a corpus of articles (avoiding the cold start problem) and performs well on a test corpus. We achieved an F-measure of 0.84 for detecting references in full-texts and an F-measure of 0.83 for finding correct matches of detected references in the da|ra dataset registry.

Data set entity recognition based on distant supervision

“…Subsequently, Duck et al (2013) applied local cues and cross-mentioned cues as features to design a rule-based entity recognition system, BioNerDS, to identify data sets and software names in medical science literature. Furthermore, Ghavimi et al (2016) proposed a semi-automatic approach based on special features extracted from data set titles to find data set references and links. Compared to manual methods, rule-based methods enable semi-automatic identification of data set entities across numerous documents.…”

The Automatic Detection of Dataset Names in Scientific Articles

“…[30]. Research on the dataset name extraction task uses a great variety of methods throughout the NER spectrum, including, but not limited to, the following: rule-based, BiLSTM-CRF and BERT [3,[6][7][8][9][10][11][12][13][14].…”

“…The advantage of BiLSTM is that it is better at predicting long sequences, predicting every word individually [41], while CRF predicts based on the joint probability of the whole sentence, making sure that the optimal sequence of tags is achieved [41][42][43]. To date, the BiLSTM-CRF based model produces the best performance on the dataset extraction task, with an F1 score of 0.85 [8].…”

“…When focusing on the articles that use a NER method for this task, it becomes clear that almost every article uses another approach and another dataset. Not only do these approaches differ, but they also deviate from the core dataset NER task, as every approach has something extra added onto it [3,[6][7][8][9][10][11][12][13][14]. This makes it hard to compare which method or component fits a task best.…”

Identifying Data Sharing and Reuse with Scholix: Potentials and Limitations

“…Mathiak and Boland 12 and Ghavimi et al. 13 explored variations in citation practices for social sciences datasets, and the latter study proposed a linked data approach to solve this issue by developing an ontology.…”