Using Ellipsis Detection and Word Similarity for Transformation of Spoken Language into Grammatically Valid Sentences

Abstract

When humans speak they often use grammatically incorrect sentences, which is a problem for grammar-based language processing methods, since they expect input that is valid for the grammar. We present two methods to transform spoken language into grammatically correct sentences. The first is an algorithm for automatic ellipsis detection, which finds ellipses in spoken sentences and searches in a combinatory categorial grammar for suitable words to fill the ellipses. The second method is an algorithm that computes the semantic similarity of two words using WordNet, which we use to find alternatives to words that are unknown to the grammar. In an evaluation, we show that the usage of these two methods leads to an increase of 38.64% more parseable sentences on a test set of spoken sentences that were collected during a human-robot interaction experiment.

A statistical approach for Non-Sentential Utterance Resolution for Interactive QA System

“…The occurrence and resolution of NSUs in a conversation have been studied in the literature and is an active area of research. However, most of the proposed approaches in the past have adopted a rule or grammar based approach [Carbonell, 1983, Fernández et al, 2005, Giuliani et al, 2014. The design of the rules or grammars in these works were motivated by the frequent patterns observed empirically which may not scale well for unseen or domain specific scenarios.…”

“…Also, note that while the NSU resolution task can be quite broad in scope and cover many aspects including ellipsis [Giuliani et al, 2014], we limit the investigation in this paper to only the Question aspect of NSU, i.e. resolving C:2 and C:3 in the example above.…”

“…Some authors have also referred to this phenomenon as Ellipsis because of the elliptical form of the NSU [Carbonell, 1983, Fernández et al, 2004, Dalrymple et al, 1991, Nielsen, 2004, Giuliani et al, 2014. While the statistical approaches have been investigated for the purpose of ellipsis detection [Fernández et al, 2004, Nielsen, 2004, Giuliani et al, 2014, it has been a common practice to use rules -syntactic or semantic -for the purpose of Ellipsis resolution [Carbonell, 1983, Dalrymple et al, 1991, Giuliani et al, 2014.…”