Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks

Aghaebrahimian, Ahmad; Cieliebak, Mark

doi:10.18653/v1/p19-1557

Cited by 6 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Artists dataset contains alternative names for music artists extracted from MusicBrains [26]. Finally, the Patent Assignee dataset contains the aliases of assignees in patent documents 2 . Table 1 All entities in the training data including true and false names are used to generate a list of most frequent n-gram characters limited to bi-, tri-, and four-grams.…”

Section: Resultsmentioning

confidence: 99%

“…Most advanced neural models today use CRFs for making inference but instead of doing feature engineering manually, they use a form of Deep Neural Network (DNN) such as Long Short Term Memory (LSTM), Convolutional Neural Network (CNN), or Gated Recurrent Unit (GRU) for automatic feature learning [2,18]. Designing a NED system by feature engineering is a highly time-consuming process, hence end-to-end neural systems capable of learning the features on their own [16,29] are more approachable systems.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Named Entity Disambiguation at Scale

Aghaebrahimian

Cieliebak

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Named Entity Disambiguation (NED) is a crucial task in many Natural Language Processing applications such as entity linking, record linkage, knowledge base construction, or relation extraction, to name a few. The task in NED is to map textual variations of a named entity to its formal name. It has been shown that parameter-less models for NED do not generalize to other domains very well. On the other hand, parametric learning models do not scale well when the number of formal names expands above the order of thousands or more. To tackle this problem, we propose a deep architecture with superior performance on NED and introduce a strategy to scale it to hundreds of thousands of formal names. Our experiments on several datasets for alias detection demonstrate that our system is capable of obtaining superior results with a large margin compared to other state-of-the-art systems.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Named Entity Disambiguation at Scale

Aghaebrahimian

Cieliebak

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use an advanced model based on Deep Learning proposed by [10]. The model uses Gated Recurrent Units (GRU) [29] for encoding the input features.…”

Section: Dl-based Classificationmentioning

confidence: 99%

“…We automatically tagged the articles in MEDLINE with these ontologies to compile a dataset for Ontology Classification (Please see Section 3). To establish a baseline for Ontology Classification over this dataset, we opted for three models: a Naive Bayes classifier, a Deep Learning(DL)-based model proposed by [10], and a Transformer-based model called PubMedBERT [11]. Before providing our detailed methods and results, we review the state-of-the-art biomedical text classification for the sake of completeness.…”

Section: Introductionmentioning

confidence: 99%

MEDOBO: 18 Million+ MEDLINE Abstracts Tagged with OBO Ontologies

Aghaebrahimian¹,

Anisimova²,

Gil³

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Motivation: The number of biomedical and life science publications is rapidly increasing, with MEDLINE alone having over 30 million citations. The massive amount of information highlights the need for biomedical text analytics to improve the accessibility of bio-data. However, the presence of multiple subdomains within life sciences poses a significant challenge. To address this, an automatic classification system is required that can annotate texts with relevant subdomains in a text analytics pipeline. This will make text processing more efficient and precise for subsequent modules in the pipeline by reducing their search space to concepts within a limited set of subdomains. The development of text classification systems for life sciences subdomains requires training data as the first requirement, but unfortunately, a high-quality and large-scale dataset for this task is lacking. Results: We present MEDOBO, a dataset comprising over 18 million MEDLINE abstracts that have been automatically labeled with ontologies from the Open Biological and Biomedical Ontology (OBO) Foundry using Medical Subject Headings (MeSH). Our dataset has achieved a conformity score of over 91% F1 when compared to an expert-annotated dataset. We have trained multiple models on MEDOBO to evaluate distribution and classification difficulty and the results indicate significant potential for enhancing multi-label text classification.

show abstract

“…Although bilingual corpus is very important for NMT, it is very scare for many low-recourse language pairs. Many language pairs and domains are suffer from the scarcity problem of bilingual corpus on machine translation [6]. Therefore, recent works attempt to mining bilingual parallel sentences from Internet which has enough monolingual corpus for many languages [7,8].…”

Section: Introductionmentioning

confidence: 99%

Mining Parallel Sentences from Internet with Multi-view Knowledge Distillation for Low-resource Language Pairs

Zhu

et al. 2023

Preprint

View full text Add to dashboard Cite

The neural machine translation (NMT), which relies on a large training data (bilingual parallel sentences, for NMT) to obtain the state-of-the-art performance, is similar with deep learning. In order to construct NMT systems, the number of parallel sentences is very important. However, these bilingual resources are scare to many low-resource language pairs. Although several works attempt to obtain bilingual parallel data from Internet, the quality and quantity of mined bilingual corpus are limited for low-resource language pairs. To address this problem, we propose the multi-view knowledge distillation model (MvKD) that use the knowledge of high-resource language pairs transfer into low-resource languages by leveraging internal language-invariant cross different languages. In particular , we treat the mining bilingual parallel sentence pair task as classifying task and use the multi-view classifier to detect bilingual parallel sentence pair. For multi-view classifier, we use two views to recognize the semantic difference of two sentences:(i) word-level representations (ii) sentence-level representations. We encode sentence-level representations to capture semantically similar of two sentences. Moreover, we encode word-level representations to capture word translations in a pair of parallel sentences to avoid the problem that semantically similar but non-parallel sentences. Experimental results demonstrate that our proposed method can significantly mine amount of bilingual corpus and improve the quality of parallel sentences. In particular, we carry out the experiments on several real-world low-resource situations and achieve excellent results.

show abstract

Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks

Cited by 6 publications

References 22 publications

Named Entity Disambiguation at Scale

Named Entity Disambiguation at Scale

MEDOBO: 18 Million+ MEDLINE Abstracts Tagged with OBO Ontologies

Mining Parallel Sentences from Internet with Multi-view Knowledge Distillation for Low-resource Language Pairs

Contact Info

Product

Resources

About