Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting: A review and empirical validation

Mandera, Paweł; Keuleers, Emmanuel; Brysbaert, Marc

doi:10.1016/j.jml.2016.04.001

Cited by 387 publications

(538 citation statements)

References 66 publications

Supporting

Mentioning

518

Contrasting

Order By: Relevance

“…A difficulty metric was computed for each of these words, by summing the z scores for frequency (Reppen et al, ), concreteness (Coltheart, ), age of acquisition (Kuperman, Stadthagen‐Gonzalez, & Brysbaert, ), and length (number of letters); frequency and age of acquisition were doubly weighted. The pairwise semantic distances between all of the 4,000 words were estimated with snaut (Mandera, Keuleers, & Brysbaert, ), a prediction‐based model of distributional semantics derived from corpora. For each word, the ten most related words output by snaut were considered as possible match items (these items were not necessarily actually closely related to the target word, given the limitations of the computational model).…”

Section: Methodsmentioning

confidence: 99%

An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia

Wilson

Yen

Eriksson

2018

Human Brain Mapping

View full text Add to dashboard Cite

Research on neuroplasticity in recovery from aphasia depends on the ability to identify language areas of the brain in individuals with aphasia. However, tasks commonly used to engage language processing in people with aphasia, such as narrative comprehension and picture naming, are limited in terms of reliability (test-retest reproducibility) and validity (identification of language regions, and not other regions). On the other hand, paradigms such as semantic decision that are effective in identifying language regions in people without aphasia can be prohibitively challenging for people with aphasia. This paper describes a new semantic matching paradigm that uses an adaptive staircase procedure to present individuals with stimuli that are challenging yet within their competence, so that language processing can be fully engaged in people with and without language impairments. The feasibility, reliability and validity of the adaptive semantic matching paradigm were investigated in sixteen individuals with chronic post-stroke aphasia and fourteen neurologically normal participants, in comparison to narrative comprehension and picture naming paradigms. All participants succeeded in learning and performing the semantic paradigm. Test-retest reproducibility of the semantic paradigm in people with aphasia was good (Dice coefficient = 0.66), and was superior to the other two paradigms. The semantic paradigm revealed known features of typical language organization (lateralization; frontal and temporal regions) more consistently in neurologically normal individuals than the other two paradigms, constituting evidence for validity. In sum, the adaptive semantic matching paradigm is a feasible, reliable and valid method for mapping language regions in people with aphasia.

show abstract

Section: Methodsmentioning

confidence: 99%

An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia

Wilson

Yen

Eriksson

2018

Human Brain Mapping

View full text Add to dashboard Cite

show abstract

“…We defined semantic similarity as the cosine between distributional vectors representing two words in a multi-dimensional lexical space. To this end, we collected pairwise estimates of semantic similarity using the Latent Semantic Analysis (LSA) from the UKWAC and SUBTLEX-UK corpus of English (available at http://zipf.ugent.be/snaut-english/, Mandera, Keuleers, & Brysbaert, in press). This application uses a 300-dimensional semantic space with CBOW embeddings, and a 6-word window for calculating co-occurrence statistics.…”

Section: Methodsmentioning

confidence: 99%

“…This application uses a 300-dimensional semantic space with CBOW embeddings, and a 6-word window for calculating co-occurrence statistics. The LSA solution for Dutch was trained on SONAR-500 and subtitle corpora, and used a 200-dimensional semantic space with CBOW embeddings and a window of 10 (available at http://zipf.ugent.be/snaut-dutch/, Mandera et al, in press). A greater LSA score indicates a greater dissimilarity between the meanings of a pair of words.…”

Section: Methodsmentioning

confidence: 99%

Surviving blind decomposition: A distributional analysis of the time-course of complex word recognition.

Schmidtke¹,

Matsuki²,

Kuperman³

2017

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

The current study addresses a discrepancy in the psycholinguistic literature about the chronology of information processing during the visual recognition of morphologically complex words. Form-thenmeaning accounts of complex word recognition claim that morphemes are processed as units of form prior to any influence of their meanings, whereas form-and-meaning models posit that recognition of complex word forms involves the simultaneous access of morphological and semantic information.The study reported here addresses this theoretical discrepancy by applying a non-parametric distributional technique of survival analysis (Reingold & Sheridan, 2014) to two behavioural measures of complex word processing. Across seven experiments reported here, this technique is employed to estimate the point in time at which orthographic, morphological and semantic variables exert their earliest discernible influence on lexical decision reaction times and eye movement fixation durations. Contrary to form-then-meaning predictions, Experiments 1-4 reveal that surface frequency is the earliest lexical variable to exert a demonstrable influence on lexical decision reaction times for English and Dutch derived words (e.g., badness; bad + -ness), English pseudo-derived words (e.g., wander; wand + -er) and morphologically simple control words (e.g., ballad; ball + -ad). Furthermore, for derived word processing across lexical decision and eye-tracking paradigms (Experiments 1-2; 5-7), semantic effects emerge early in the time-course of word recognition, and their effects either precede or emerge simultaneously with morphological effects. These results are not consistent with the premises of the form-then-meaning view of complex word recognition, but are convergent with a form-and-meaning account of complex word recognition.

show abstract

“…Several studies have already shown correspondence between priming magnitude and VSM measures of relation such as cosine similarity or neighbor rank (Mandera et al, 2016;Lapesa and Evert, 2013;Jones et al, 2006;Padó and Lapata, 2007;Herdagdelen et al, 2009;McDonald and Brew, 2004). These positive results suggest that some of the implicit relation structure in the human brain is already reflected in current vector space models, and that it is in fact feasible to evaluate relation structure of VSMs by testing their ability to predict this implicit human measure.…”

Section: Semantic Primingmentioning

confidence: 96%

Evaluating vector space models using human semantic priming results

Ettinger

Linzen

2016

Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP

View full text Add to dashboard Cite

Vector space models of word representation are often evaluated using human similarity ratings. Those ratings are elicited in explicit tasks and have well-known subjective biases. As an alternative, we propose evaluating vector spaces using implicit cognitive measures. We focus in particular on semantic priming, exploring the strengths and limitations of existing datasets, and propose ways in which those datasets can be improved.

show abstract

Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting: A review and empirical validation

Cited by 387 publications

References 66 publications

An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia

An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia

Surviving blind decomposition: A distributional analysis of the time-course of complex word recognition.

Evaluating vector space models using human semantic priming results

Contact Info

Product

Resources

About