New Word Analogy Corpus for Exploring Embeddings of Czech Words

Svoboda, Lukás̆; Brychcín, Tomáš

doi:10.1007/978-3-319-75477-2_6

Cited by 13 publications

(17 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Word2vec vectors for Czech experiments are trained on Czech Wikipedia (Svoboda and Brychcín, 2016). For the English experiments we utilize the standard vectors trained on part of Google News dataset (Mikolov et al, 2013b).…”

Section: Tools and Corporamentioning

confidence: 99%

Word Embeddings for Multi-label Document Classification

Lenc

Král

2017

RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning

View full text Add to dashboard Cite

In this paper, we analyze and evaluate word embeddings for representation of longer texts in the multi-label document classification scenario. The embeddings are used in three convolutional neural network topologies. The experiments are realized on the CzechČTK and English Reuters-21578 standard corpora. We compare the results of word2vec static and trainable embeddings with randomly initialized word vectors. We conclude that initialization does not play an important role for classification. However, learning of word vectors is crucial to obtain good results.

show abstract

Section: Tools and Corporamentioning

confidence: 99%

Word Embeddings for Multi-label Document Classification

Lenc

Král

2017

RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning

View full text Add to dashboard Cite

show abstract

“…This is the reason why we bring up the comparison with highly inflected language. In [33] and [32] has been shown that there is a space for the performance improvement of current state-of-the-art word embedding models on languages from Slavic families. More information about individual section of Czech word analogy corpus is described in [33].…”

Section: Discussionmentioning

confidence: 99%

“…Such a question can be answered with a simple equation: vec(king) − vec(queen) = vec(man) − vec(woman). We evaluate on English and Czech word analogy datasets, proposed by [21] and [33], respectively. The word-phrases were excluded from original datasets, resulting in 8869 semantic and 10,675 syntactic questions for English (19,544 in total), and 6018 semantic.…”

Section: Trainingmentioning

confidence: 99%

Enriching Word Embeddings with Global Information and Testing on Highly Inflected Language

Svoboda

Brychcín

2019

CyS

Self Cite

View full text Add to dashboard Cite

In this paper we evaluate our new approach based on the Continuous Bag-of-Words and Skip-gram models enriched with global context information on highly inflected Czech language and compare it with English results. As a source of information we use Wikipedia, where articles are organized in a hierarchy of categories. These categories provide useful topical information about each article. Both models are evaluated on standard word similarity and word analogy datasets. Proposed models outperform other word representation methods when similar size of training data is used. Model provide similar performance especially with methods trained on much larger datasets.

show abstract

“…Finally, most evaluations of word2vec embeddings focus on English, with notable exceptions (Köper et al, 2015;Berardi et al, 2015;Svoboda and Brychcín, 2018;Venekoski and Vankka, 2017;Rodrigues et al, 2016;Chen et al, 2015;Grave et al, 2018). However, these are translations of word similarity tasks and share the weaknesses of their English language counterparts.…”

Section: Related Workmentioning

confidence: 99%

One of these words is not like the other: a reproduction of outlier identification using non-contextual word representations

Andersen¹,

Bertelsen²,

Schou³

et al. 2020

Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems

View full text Add to dashboard Cite

Word embeddings are an active topic in the NLP research community. State-of-the-art neural models achieve high performance on downstream tasks, albeit at the cost of computationally expensive training. Cost aware solutions require cheaper models that still achieve good performance. We present several reproduction studies of intrinsic evaluation tasks that evaluate non-contextual word representations in multiple languages. Furthermore, we present 50-8-8, a new data set for the outlier identification task, which avoids limitations of the original data set, such as ambiguous words, infrequent words, and multi-word tokens, while increasing the number of test cases. The data set is expanded to contain semantic and syntactic tests and is multilingual (English, German, and Italian).We provide an in-depth analysis of word embedding models with a range of hyperparameters. Our analysis shows the suitability of different models and hyper-parameters for different tasks and the greater difficulty of representing German and Italian languages.

show abstract

New Word Analogy Corpus for Exploring Embeddings of Czech Words

Cited by 13 publications

References 12 publications

Word Embeddings for Multi-label Document Classification

Word Embeddings for Multi-label Document Classification

Enriching Word Embeddings with Global Information and Testing on Highly Inflected Language

One of these words is not like the other: a reproduction of outlier identification using non-contextual word representations

Contact Info

Product

Resources

About