A prosody-based vector-space model of dialog activity for information retrieval

Abstract

Search in audio archives is a challenging problem. Using prosodic information to help find relevant content has been proposed as a complement to word-based retrieval, but its utility has been an open question. We propose a new way to use prosodic information in search, based on a vector-space model, where each point in time maps to a point in a vector space whose dimensions are derived from numerous prosodic features of the local context. Point pairs that are close in this vector space are frequently similar, not only in terms of the dialog activities, but also in topic. Using proximity in this space as an indicator of similarity, we built support for a query-by-example function. Searchers were happy to use this function, and it provided value on a large testset. Prosody-based retrieval did not perform as well as word-based retrieval, but the two sources of information were often non-redundant and in combination they sometimes performed better than either separately.

An ontology-improved vector space model for semantic retrieval

“…The VSM could be regarded as the earliest semantic retrieval model (Salton et al, 1975). It is widely used in various kinds of IR (Chew et al, 2011;Ward et al, 2015), document ranking (Gupta et al, 2013;Lakshminarayana, 2003) and text mining (Carrera-Trejo et al, 2015;Costa and Lima, 2015). However, the model defines that the keywords of one document are mutually independent of another.…”

Using Prosody to Spot Location Mentions

“…These were designed to be robust, as is necessary for spontaneous speech in general, and especially for Switchboard, given its varied audio quality [18]. Like other feature sets [6], this feature set has been shown in previous work to be informative regarding many semantic and pragmatic functions [18,24,19,21].…”

Interactional and pragmatics-related prosodic patterns in Mandarin dialog

“…In European languages prosody is known to convey many pragmatic functions, and prosody-pragmatics mappings have been exploited in many applications, including turn-taking, user modeling, information retrieval, and speech recognition [1,2,3,4,5]. However tone languages may use prosody less for pragmatic purposes, and to the extent that this is the case, many techniques that use prosody will be less effective.…”