A structured vector space model for word meaning in context

Abstract

We address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing models for this task do not take syntactic structure sufficiently into account. We present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words' argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.

A Formal Approach to Linking Logical Form and Vector-Space Lexical Semantics

“…We do this by computing vector representations for word meaning in context, and predicting rule applicability based on these contextspecific vectors. We follow the literature on vector space representations for word meaning in context (Erk and Padó, 2008;Thater et al, 2010;Reisinger and Mooney, 2010;Dinu and Lapata, 2010;Van de Cruys et al, 2011) in assuming that a word's context-specific meaning is a function of its out-of-context representation and the context. The context may consist of a single item or multiple items, and (syntactic or semantic) relations to the target word may also play a role (Erk and Padó, 2008;Thater et al, 2010;Van de Cruys et al, 2011).…”

“…This follows common practice in vector space models of word meaning in context of computing a context-specific representation of the target, but not the paraphrase candidate. But if the paraphrase candidate is polysemous, it may be useful to compute a representation for it that is also specific to the sentence context at hand (Erk and Padó, 2010). We can do this by defining a lexical mapping γ P,G specific to predicate P and formula G by γ P,G (Q) = α (Q), κ(P, G) .…”

“…They, therefore, tend to have high precision at the cost of low recall (Bos and Markert, 2005). meaning have been used successfully to judge paraphrase appropriateness by representing the meaning of a word in context as a point in a high-dimensional semantics space (Erk and Padó, 2008;Thater et al, 2010;Reisinger and Mooney, 2010;Dinu and Lapata, 2010;Van de Cruys et al, 2011). However, these models only address word meaning, and do not address the question of providing meaning representations for complete sentences.…”

“…In a suitably constructed distributional representation, distributional similarity between two words or expressions A and B indicates that B can be substituted for A in text (Lin and Pantel, 2001). This can be described through an inference rule A → B. Distributional information can also be used to determine the degree η to which the rule applies in a given sentence context (Szpektor et al, 2008;Mitchell and Lapata, 2008;Erk and Padó, 2008;Thater et al, 2010;Reisinger and Mooney, 2010;Dinu and Lapata, 2010;Van de Cruys et al, 2011). This degree η can be used as a weight on the inference rule A → B.…”

“…They can be learned in an unsupervised fashion. Recently distributional models have been used to predict the applicability of paraphrases in context (Erk and Padó, 2008;Thater et al, 2010;Reisinger and Mooney, 2010;Dinu and Lapata, 2010;Van de Cruys et al, 2011). For example, in "The spill left a stain", result in is a better paraphrase for leave than flee, because of the context of spill and stain.…”