Beyond the Benchmarks: Toward Human-Like Lexical Representations

Stevenson, Suzanne; Merlo, Paola

doi:10.3389/frai.2022.796741

Cited by 3 publications

(2 citation statements)

References 148 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though they are loosely inspired by neural connections in the brain ( Perconti & Plebe, 2020 ; Rumelhart et al, 1986 ; Rumelhart & McClelland, 1987 ), artificial neural network models are considered neurobiologically unrealistic models both at the level of implementation in the human brain ( McClelland & Botvinick, 2020 ; Rosenbaum, 2022 ; Thomas & McClelland, 2008 ) as well as in regard to their functional similarity to human language processing ( Arehalli et al, 2022 ; Arehalli & Linzen, 2020 ) and learning ( Stevenson & Merlo, 2022 ; Warstadt & Bowman, 2022 ). This notwithstanding, the engagement in constant next-word prediction is an important functional property shared between language models and human sentence processing ( Goldstein et al, 2022 ).…”

Section: Discussionmentioning

confidence: 99%

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Huber,

Sauppe,

Isasi-Isasmendi

et al. 2024

Neurobiology of Language

Self Cite

View full text Add to dashboard Cite

Language models based on artificial neural networks increasingly capture key aspects of how humans process sentences. Most notably, model-based surprisals predict event-related potentials such as N400 amplitudes during parsing. Assuming that these models represent realistic estimates of human linguistic experience, their success in modelling language processing raises the possibility that the human processing system relies on no other principles than the general architecture of language models and on sufficient linguistic input. Here, we test this hypothesis on N400 effects observed during the processing of verb-final sentences in German, Basque, and Hindi. By stacking Bayesian generalised additive models, we show that, in each language, N400 amplitudes and topographies in the region of the verb are best predicted when model-based surprisals are complemented by an Agent Preference principle that transiently interprets initial role-ambiguous NPs as agents, leading to reanalysis when this interpretation fails. Our findings demonstrate the need for this principle independently of usage frequencies and structural differences between languages. The principle has an unequal force, however. Compared to surprisal, its effect is weakest in German, stronger in Hindi, and still stronger in Basque. This gradient is correlated with the extent to which grammars allow unmarked NPs to be patients, a structural feature that boosts reanalysis effects. We conclude that language models gain more neurobiological plausibility by incorporating an Agent Preference. Conversely, theories of human processing profit from incorporating surprisal estimates in addition to principles like the Agent Preference, which arguably have distinct evolutionary roots.

show abstract

Section: Discussionmentioning

confidence: 99%

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Huber,

Sauppe,

Isasi-Isasmendi

et al. 2024

Neurobiology of Language

Self Cite

View full text Add to dashboard Cite

show abstract

“…Producing sentence representations is a non-trivial issue, mainly because of the structural grammatical and semantic relations they express and their varying complexity and length (Stevenson and Merlo, 2022). The deep learning framework has allowed for a variety of elegant solutions to explicitly learn sentence representations or to induce them as a side-effect or modeling of a more complex problem (Mikolov et al, 2013;Pennington et al, 2014;Bojanowski et al, 2017;Peters et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Grammatical information in BERT sentence embeddings as two-dimensional arrays

Năstase¹,

Merlo²

2023

Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023)

Self Cite

View full text Add to dashboard Cite

Sentence embeddings induced with various transformer architectures encode much semantic and syntactic information in a distributed manner in a one-dimensional array. We investigate whether specific grammatical information can be accessed in these distributed representations. Using data from a task developed to test rule-like generalizations, our experiments on detecting subject-verb agreement yield several promising results. First, we show that while the usual sentence representations encoded as onedimensional arrays do not easily support extraction of rule-like regularities, a two-dimensional reshaping of these vectors allows various learning architectures to access such information. Next, we show that various architectures can detect patterns in these two-dimensional reshaped sentence embeddings and successfully learn a model based on smaller amounts of simpler training data, which performs well on more complex test data. This indicates that current sentence embeddings contain information that is regularly distributed, and which can be captured when the embeddings are reshaped into higher dimensional arrays. Our results cast light on representations produced by language models and help move towards developing fewshot learning approaches.

show abstract

Systematic Approach for Measuring Semantic Relatedness between Ontologies

Elfaki

Alfaifi

2023

Electronics

View full text Add to dashboard Cite

Measuring ontology matching is a critical issue in knowledge engineering and supports knowledge sharing and knowledge evolution. Recently, linguistic scientists have defined semantic relatedness as being more significant than semantic similarities in measuring ontology matching. Semantic relatedness is measured using synonyms and hypernym–hyponym relationships. In this paper, a systematic approach for measuring ontology semantic relatedness is proposed. The proposed approach is developed with a clear and fully described methodology, with illustrative examples used to demonstrate the proposed approach. The relatedness between ontologies has been measured based on class level by using lexical features, defining semantic similarity of concepts based on hypernym–hyponym relationships. For evaluating our proposed approach against similar works, benchmarks are generated using five properties: related meaning features, lexical features, providing technical descriptions, proving applicability, and accuracy. Technical implementation is carried out in order to demonstrate the applicability of our approach. The results demonstrate an achieved accuracy of 99%. The contributions are further highlighted by benchmarking against recent related works.

show abstract

Beyond the Benchmarks: Toward Human-Like Lexical Representations

Cited by 3 publications

References 148 publications

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Grammatical information in BERT sentence embeddings as two-dimensional arrays

Systematic Approach for Measuring Semantic Relatedness between Ontologies

Contact Info

Product

Resources

About