Using experiential optimization to build lexical representations

Johns, Brendan T.; Jones, Michael N.; Mewhort, D. J. K.

doi:10.3758/s13423-018-1501-2

Cited by 32 publications

(45 citation statements)

References 110 publications

(203 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, a reviewer suggested an additional note of caution in such modeling enterprises. Representations in semantic space models are highly dependent on the choice of corpus (Johns et al, 2019;Mandera et al, 2017) -this is illustrated in Supplementary Materials A where comparison between BEAGLE on two different corpora reveals different abilities to predict human performance in semantic ratings and synonym selection, and in Supplementary Materials C where we demonstrate different weight parameters of the global similarity metrics when the TASA corpus is used instead of our novels corpus. In addition, Morton and Polyn (2016) demonstrated how results in free recall are contingent both on the choice of semantic space model and retrieval model.…”

Section: Matching Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Global semantic similarity effects in recognition memory: Insights from BEAGLE representations and the diffusion decision model

Osth

Shabahang

Mewhort

et al. 2020

Journal of Memory and Language

View full text Add to dashboard Cite

We would like to thank Amy Criss and Asli Kiliç for generously sharing their data and Brendan Johns for providing the datasets of similarity and relatedness judgments. This work was supported by an ARC Discovery Early Career Research Award (DE170100106) awarded to Adam Osth. BEAGLE vectors, datasets, and model code can be found on https://osf.io/gtdqf/ GLOBAL SEMANTIC SIMILARITY EFFECTS 2 AbstractRecognition memory models posit that false alarm rates increase as the global similarity between the probe cue and the contents of memory is increased. Global similarity predictions have been commonly tested using category length designs where it has been found that false alarm rates increase as the number of studied items from a common category is increased. In this work, we explored global similarity predictions within unstructured lists of words using representations from the BEAGLE model (Jones & Mewhort, 2007). BEAGLE differs from traditional semantic space models in that it contains two types of representations: item vectors, which encode unordered co-occurrence, and order vectors, in which words are similar to the extent to which they are share neighboring words in the same relative positions. Global similarity among item and order vectors was regressed onto drift rates in the diffusion decision model (DDM: Ratcliff, 1978), which unifies both response times and accuracy. We implemented this model in a hierarchical Bayesian framework across seven datasets with lists composed of unrelated words. Results indicated clear deficits due to global similarity among item vectors, but only a minimal impact of global similarity among the order vectors.

show abstract

Section: Matching Modelsmentioning

confidence: 99%

“…Our BEAGLE representations were constructed from a corpus of novels, which has been used in previous publications (Johns & Jamieson, 2018;Johns, Jones, & Mewhort, 2019;Mewhort et al, 2018). The corpus contained 39,076 unique words in 10,238,600…”

Section: Constructing Word Representations With Beaglementioning

confidence: 99%

Global semantic similarity effects in recognition memory: Insights from BEAGLE representations and the diffusion decision model

Osth

Shabahang

Mewhort

et al. 2020

Journal of Memory and Language

View full text Add to dashboard Cite

show abstract

“…Younger and older adults differ in occupational status [37], social networks [38], and their use of the Internet and social media [39]. These differences in experience further contribute to shaping the contents of younger and older adults' lexical and semantic representations [40]. Regrettably, the extent to which differences in the amount and content of information exposed to younger and older adults determines their lexical and semantic representations and cognitive performance remains largely unexplored.…”

Section: Different Environmentsmentioning

confidence: 99%

New Perspectives on the Aging Lexicon

Wulff

Deyne

Jones

et al. 2019

Trends in Cognitive Sciences

Self Cite

122

126

View full text Add to dashboard Cite

The field of cognitive aging has seen considerable advances in describing the linguistic and semantic changes that happen during the adult life span to uncover the structure of the mental lexicon (i.e., the mental repository of lexical and conceptual representations). Nevertheless, there is still debate concerning the sources of these changes, including the role of environmental exposure and several cognitive mechanisms associated with learning, representation, and retrieval of information. We review the current status of research in this field and outline a framework that promises to assess the contribution of both ecological and psychological aspects to the aging lexicon. Cognitive Aging and the Mental Lexicon There is consensus in the cognitive sciences that human development extends well beyond childhood and adolescence, and there has been remarkable empirical progress in the field of cognitive aging in past decades [1]. Nevertheless, the role of environmental and cognitive factors in age-related changes in the structure and processing of lexical and semantic representations (see Glossary) is still under debate. For example, age-related memory decline is commonly attributed to a decline in cognitive abilities [2,3], yet some researchers have proposed that massive exposure to language over the course of one's life leads to knowledge gains that may contribute to, if not fully account for, age-related memory deficits [4-6]. We argue that to resolve such debates we require an interdisciplinary approach that captures how information exposure across adulthood may change the way that we acquire, represent, and recall information. We summarize recent developments in the field (Figure 1, Table 1) and propose a conceptual framework (Figure 2, Key Figure) and associated research agenda that argues for combining ecological analyses, formal modeling, and large-scale empirical studies to shed light on the contents, structure, and neural basis of the aging mental lexicon in both health and disease. Mental Lexicon: Aging and Cognitive Performance The mental lexicon can be thought of as a repository of lexical and conceptual representations, composed of organized networks of semantic, phonological, orthographic, morphological, and other types of information [7]. The cognitive sciences have provided considerable knowledge about the computational (Box 1; [8-11]) and neural basis (Box 2; [12,13]) of lexical and semantic cognition, and there has been considerable interest in how such aspects of cognition change across adulthood and aging [14,15]. Past work on the aging lexicon emphasized the amount of information acquired across the life span (e.g., vocabulary gains across adulthood; [15]); however, new evaluations using graphbased approaches suggest that both quantity and structural aspects of representations differ between individuals [16] and change across the life span [17-19]. Such insights were gathered, for example, from a large-scale analysis of free association data from thousands of individuals [17], ranging from 10 to ...

show abstract

“…By manipulating the parameter t , the subsampling distribution is changed. The correct setting for this parameter is likely corpus dependent, as there are significant deviations in frequency distributions by type of corpus used (see Johns & Jamieson, and Johns, Jones, & Mewhort, , for examples).…”

Section: Modeling Frameworkmentioning

confidence: 99%

“…We used a corpus of 20 million sentences, derived by combining Wikipedia articles and non‐fiction books (Johns, Jones, & Mewhort, ; Johns et al., in press). The corpus consists of approximately 120 million words.…”

Section: Modeling Frameworkmentioning

confidence: 99%

The Role of Negative Information in Distributional Semantic Learning

2019

Self Cite

View full text Add to dashboard Cite

Distributional models of semantics learn word meanings from contextual co‐occurrence patterns across a large sample of natural language. Early models, such as LSA and HAL (Landauer & Dumais, 1997; Lund & Burgess, 1996), counted co‐occurrence events; later models, such as BEAGLE (Jones & Mewhort, 2007), replaced counting co‐occurrences with vector accumulation. All of these models learned from positive information only: Words that occur together within a context become related to each other. A recent class of distributional models, referred to as neural embedding models, are based on a prediction process embedded in the functioning of a neural network: Such models predict words that should surround a target word in a given context (e.g., word2vec; Mikolov, Sutskever, Chen, Corrado, & Dean, 2013). An error signal derived from the prediction is used to update each word's representation via backpropagation. However, another key difference in predictive models is their use of negative information in addition to positive information to develop a semantic representation. The models use negative examples to predict words that should not surround a word in a given context. As before, an error signal derived from the prediction prompts an update of the word's representation, a procedure referred to as negative sampling. Standard uses of word2vec recommend a greater or equal ratio of negative to positive sampling. The use of negative information in developing a representation of semantic information is often thought to be intimately associated with word2vec's prediction process. We assess the role of negative information in developing a semantic representation and show that its power does not reflect the use of a prediction mechanism. Finally, we show how negative information can be efficiently integrated into classic count‐based semantic models using parameter‐free analytical transformations.

show abstract

Using experiential optimization to build lexical representations

Cited by 32 publications

References 110 publications

Global semantic similarity effects in recognition memory: Insights from BEAGLE representations and the diffusion decision model

Global semantic similarity effects in recognition memory: Insights from BEAGLE representations and the diffusion decision model

New Perspectives on the Aging Lexicon

The Role of Negative Information in Distributional Semantic Learning

Contact Info

Product

Resources

About