Mapping Lexical Knowledge to Distributed Models for Ontology Concept Invention

Vimercati, Manuel; Bianchi, Federico; Soto, Mauricio; Palmonari, Matteo

doi:10.1007/978-3-030-35166-3_40

Cited by 1 publication

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Word vectors are also used in topic models (Das, Zaheer, and Dyer 2015), to capture prior knowledge about word relatedness. Finally, in the context of the Semantic Web, word vectors have been used for ontology alignment (Kolyvakis, Kalousis, and Kiritsis 2018), concept invention (Vimercati et al 2019) and for automatically completing ontologies (Li, Bouraoui, and Schockaert 2019), among others.…”

Section: Introductionmentioning

confidence: 99%

Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings

Li¹,

Bouraoui²,

Camacho-Collados³

et al. 2020

Preprint

View full text Add to dashboard Cite

While the success of pre-trained language models has largely eliminated the need for high-quality static word vectors in many NLP applications, static word vectors continue to play an important role in tasks where word meaning needs to be modelled in the absence of linguistic context. In this paper, we explore how the contextualised embeddings predicted by BERT can be used to produce high-quality word vectors for such domains, in particular related to knowledge base completion, where our focus is on capturing the semantic properties of nouns. We find that a simple strategy of averaging the contextualised embeddings of masked word mentions leads to vectors that outperform the static word vectors learned by BERT, as well as those from standard word embedding models, in property induction tasks. We notice in particular that masking target words is critical to achieve this strong performance, as the resulting vectors focus less on idiosyncratic properties and more on general semantic properties. Inspired by this view, we propose a filtering strategy which is aimed at removing the most idiosyncratic mention vectors, allowing us to obtain further performance gains in property induction.

show abstract