Problems Caused by Semantic Drift in WordNet Synset Construction

Bakay, Özge; Ergelen, Özlem; Yıldız, Olcay Taner

doi:10.1109/ubmk.2019.8907181

Cited by 4 publications

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A neural language model trained on a dataset dating from 1988 or 2018 can therefore fail to perform adequately in tasks ranging from question answering to dialogue because the model is built on data that no longer represents current language. Several authors have studied the problems of semantic drift in Wordnet and other training sets [5,68,54] and proposed ways to address and mitigate it from a technical perspective [32,43], but these approaches are rarely incorporated in modern NLP models, which can be trained on large corpora reflecting decades of language data, including books from previous centuries [6].…”

Section: Technical Issuesmentioning

confidence: 99%

A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication

Luccioni,

Corry,

Sridharan

et al. 2021

Preprint

View full text Add to dashboard Cite

What happens when a machine learning dataset is deprecated for legal, ethical, or technical reasons, but continues to be widely used? In this paper, we examine the public afterlives of several prominent deprecated or redacted datasets, including ImageNet, 80 Million Tiny Images, MS-Celeb-1M, Duke MTMC, Brainwash, and HRT Transgender, in order to inform a framework for more consistent, ethical, and accountable dataset deprecation. Building on prior research [27,35,61], we find that there is a lack of consistency, transparency, and centralized sourcing of information on the deprecation of datasets, and as such, these datasets and their derivatives continue to be cited in papers and circulate online. These datasets that never die -which we term "zombie datasets" -continue to inform the design of production-level systems, causing technical, legal, and ethical challenges; in so doing, they risk perpetuating the harms that prompted their supposed withdrawal, including concerns around bias, discrimination, and privacy. Based on this analysis, we propose a Dataset Deprecation Framework that includes considerations of risk, mitigation of impact, appeal mechanisms, timeline, post-deprecation protocol, and publication checks that can be adapted and implemented by the machine learning community. Drawing on work on datasheets and checklists [29,48], we further offer two sample dataset deprecation sheets and propose a centralized repository that tracks which datasets have been deprecated and could be incorporated into the publication protocols of venues like NeurIPS.

show abstract

Section: Technical Issuesmentioning

confidence: 99%

A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication

Luccioni,

Corry,

Sridharan

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Integrating Turkish Wordnet KeNet to Princeton WordNet: The Case of One-to-Many Correspondences

Bakay

Ergelen

Yıldız

2019

2019 Innovations in Intelligent Systems and Applications Conference (ASYU)

View full text Add to dashboard Cite

Confidence Indexing of Automated Detected Synsets: A Case Study on Contemporary Turkish Dictionary

TuranErhan

OrhanUmut

2021

ACM Trans. Asian Low-Resour. Lang. Inf. Process.

View full text Add to dashboard Cite

In this study, a novel confidence indexing algorithm is proposed to minimize human labor in controlling the reliability of automatically extracted synsets from a non-machine-readable monolingual dictionary. Contemporary Turkish Dictionary of Turkish Language Association is used as the monolingual dictionary data. First, the synonym relations are extracted by traditional text processing methods from dictionary definitions and a graph is prepared in Lemma-Sense network architecture. After each synonym relation is labeled by a proper confidence index, synonym pairs with desired confidence indexes are analyzed to detect synsets with a spanning tree-based method. This approach can label synsets with one of three cumulative confidence levels (CL-1, CL-2, and CL-3). According to the confidence levels, synsets are compared with KeNet which is the only open access Turkish Wordnet. Consequently, while most matches with the synsets of KeNet is determined in CL-1 and CL-2 confidence levels, the synsets determined at CL-3 level reveal errors in the dictionary definitions. This novel approach does not find only the reliability of automatically detected synsets, but it can also point out errors of detected synsets from the dictionary.

show abstract

Problems Caused by Semantic Drift in WordNet Synset Construction

Cited by 4 publications

References 1 publication

A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication

A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication

Integrating Turkish Wordnet KeNet to Princeton WordNet: The Case of One-to-Many Correspondences

Confidence Indexing of Automated Detected Synsets: A Case Study on Contemporary Turkish Dictionary

Contact Info

Product

Resources

About