Open System Categorical Quantum Semantics in Natural Language Processing

Abstract

Originally inspired by categorical quantum mechanics (Abramsky and Coecke, LiCS'04), the categorical compositional distributional model of natural language meaning of Coecke, Sadrzadeh and Clark provides a conceptually motivated procedure to compute the meaning of a sentence, given its grammatical structure within a Lambek pregroup and a vectorial representation of the meaning of its parts. The predictions of this first model have outperformed that of other models in mainstream empirical language processing tasks on large scale data. Moreover, just like CQM allows for varying the model in which we interpret quantum axioms, one can also vary the model in which we interpret word meaning.In this paper we show that further developments in categorical quantum mechanics are relevant to natural language processing too. Firstly, Selinger's CPM-construction allows for explicitly taking into account lexical ambiguity and distinguishing between the two inherently different notions of homonymy and polysemy. In terms of the model in which we interpret word meaning, this means a passage from the vector space model to density matrices. Despite this change of model, standard empirical methods for comparing meanings can be easily adopted, which we demonstrate by a small-scale experiment on real-world data. This experiment moreover provides preliminary evidence of the validity of our proposed new model for word meaning.Secondly, commutative classical structures as well as their non-commutative counterparts that arise in the image of the CPM-construction allow for encoding relative pronouns, verbs and adjectives, and finally, iteration of the CPM-construction, something that has no counterpart in the quantum realm, enables one to accommodate both entailment and ambiguity.

Hyper-decoherence in Density Hypercubes

“…This work also explores two closely related theories from the literature, those of double dilation and double mixing, developed to describe quantum-like aspects of ambiguity in natural language processing [4,17]. It was originally believed [13] that density hypercubes and double dilation coincided: we show that not to be the case.…”

“…that all maps in the form (4) can also be put in the form (3). Both double dilation and double mixing have found application to the modelling of ambiguity in natural language processing [4,17].…”

Composing Conversational Negation

“…vector embeddings in machine learning) within one compositional framework that enables one to compute the meaning of a sentence from the meanings of its words. To achieve this it exploits the common compact closed categorical structure, be it of vectors and linear maps, or of positive operators and completely positive maps [2,31]. The DisCoCirc framework [5] improved on DisCoCat, (1) by enabling one to compose sentences into larger text, just as gates are composed in circuits; (2) by allowing meanings to evolve as that text evolves; (3) by specifying the sentence type as the tensored spaces of those entities that evolve.…”

CPM Categories for Galois Extensions

“…we recover the folding of density hypercubes [21,23], double dilation [37] and double mixing [2,9,29]. In all three cases the environment structure contains the caps from the original CPM construction (i.e.…”

“…This generalisation expands upon the inherent symmetries of the original CPM construction to produce categories with more exotic properties, captured by an essential invariance under the action of a group of monoidal autofunctors. These categories have found uses in quantum natural language processing, where they capture multiple degrees of linguistic ambiguity [2,29,9,37], and in CQM, where they have been found to exhibit higher-order interference, hyper-decoherence and a rich phase group structure [21,23].…”