Comparison of Representations of Named Entities for Document Classification

Pivovarova, Lidia; Yangarber, Roman

doi:10.18653/v1/w18-3008

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scalar δ i is equal to the average of D( x i s , x i t ) 2 over all data points. For rcv1(subset1) text classification dataset, we obtain the vectorial representation based on the word-embedding used by [30]. For MTS datasets, we compute each K i via employing the global alignment kernel (GAK) [31] for the i-th dimension of the time-series.…”

Section: Methodsmentioning

confidence: 99%

Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection

Hosseini

Hammer

2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Prototype-based methods are of the particular interest for domain specialists and practitioners as they summarize a dataset by a small set of representatives. Therefore, in a classification setting, interpretability of the prototypes is as significant as the prediction accuracy of the algorithm. Nevertheless, the state-of-the-art methods make inefficient trade-offs between these concerns by sacrificing one in favor of the other, especially if the given data has a kernel-based (or multiple-kernel) representation. In this paper, we propose a novel interpretable multiple-kernel prototype learning (IMKPL) to construct highly interpretable prototypes in the feature space, which are also efficient for the discriminative representation of the data. Our method focuses on the local discrimination of the classes in the feature space and shaping the prototypes based on condensed class-homogeneous neighborhoods of data. Besides, IMKPL learns a combined embedding in the feature space in which the above objectives are better fulfilled. When the base kernels coincide with the data dimensions, this embedding results in a discriminative features selection. We evaluate IMKPL on several benchmarks from different domains which demonstrate its superiority to the related state-of-the-art methods regarding both interpretability and discriminative representation. * Preprint of the publication [1], as provided by the authors. The final publication is available at https://dl.acm.org/proceedings.cfm

show abstract

Section: Methodsmentioning

confidence: 99%

Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection

Hosseini

Hammer

2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…Various attempts have been made to learn semantic representation from different aspects such as entities (Pivovarova and Yangarber, 2018) and abstract (Kim and Gil, 2019) etc. Our hypothesis is that entities can provide a knowledge backbone and abstract can provide an overall summary which should be modeled and learned interactively together.…”

Section: Semantic Information Learningmentioning

confidence: 99%

Content- and Topology-Aware Representation Learning for Scientific Multi-Literature

Zhang,

Song,

Kang

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Representation learning forms an essential building block in the development of natural language processing architectures. To date, mainstream approaches focus on learning textual information at the sentence-or document-level, unfortunately, overlooking the inter-document connections. This omission decreases the potency of downstream applications, particularly in multi-document settings. To address this issue, embeddings equipped with latent semantic and rich relatedness information are needed. In this paper, we propose SMRC 2 , which extends representation learning to the multi-document level. Our model jointly learns latent semantic information from content and rich relatedness information from topological networks. Unlike previous studies, our work takes multi-document as input and integrates both semantic and relatedness information using a shared space via language model and graph structure. Our extensive experiments confirm the superiority and effectiveness of our approach. To encourage further research in scientific multi-literature representation learning, we will release our code and a new dataset from the biomedical domain 1 .

show abstract

“…While they reported use of NEs improve the accuracy of document classification, the contribution to subword-based neural network models was not investigated. Pivovarova and Yangarber (2018) compared the representation of NEs for neural network-based models in document classification task. They reported that replacing tokens of named entities with special tokens representing NE categories does not improve the accuracy of document classification.…”

Section: Related Workmentioning

confidence: 99%

Improving Neural Language Processing with Named Entities

Matsushita¹,

Makino

Iwakura

2021

Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Me

View full text Add to dashboard Cite

Pretraining-based neural network models have demonstrated state-of-the-art (SOTA) performances on natural language processing (NLP) tasks. The most frequently used sentence representation for neural-based NLP methods is a sequence of subwords that is different from the sentence representation of non-neural methods that are created using basic NLP technologies, such as part-of-speech (POS) tagging, named entity (NE) recognition, and parsing. Most neural-based NLP models receive only vectors encoded from a sequence of subwords obtained from an input text. However, basic NLP information, such as POS tags, NEs, parsing results, etc, cannot be obtained explicitly from only the large unlabeled text used in pretraining-based models. This paper explores use of NEs on two Japanese tasks; document classification and headline generation using Transformer-based models, to reveal the effectiveness of basic NLP information. The experimental results with eight basic NEs and approximately 200 extended NEs show that NEs improve accuracy although a large pretrainingbased model trained using 70 GB text data was used.

show abstract

Comparison of Representations of Named Entities for Document Classification

Cited by 3 publications

References 12 publications

Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection

Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection

Content- and Topology-Aware Representation Learning for Scientific Multi-Literature

Improving Neural Language Processing with Named Entities

Contact Info

Product

Resources

About