"Show and Tell": Using Semantically Processable Prosodic Markers for Spatial Expressions in an HCI System for Consumer Complaints

Abstract

The observed relation between prosodic information and the degree of precision and lack of ambiguity is attempted to be integrated in the processing of the user's spoken input in the CitizenShield ("POLIAS") system for consumer complaints for commercial products. The prosodic information contained in the spoken descriptions provided by the consumers is attempted to be preserved with the use of semantically processable markers, classifiable within an Ontological Framework and signalizing prosodic prominence in the speakers spoken input. Semantic processability is related to the reusability and/or extensibility of the present system to multilingual applications or even to other types of monolingual applications.

Word category and prosodic emphasis in dialog modules of speech technology applications

“…The behavior of word categories in Modern Greek is observed during the prosodic modelling of utterances recorded in a studio for the construction of the speech output produced by a Conversational Agent in a dialog system for consumer complaints (CitizenShield System, www.polias.gr), labelled "CSh System", and for a dialog-module of a speech-recognition in a system for Senior Citizens which activates and monitors house appliances and the general room environment (SOPRANO Project, www.soprano-ip.org), labelled "SPR System". The acceptable prosody to the target user-groups, constituting the general public in the CSh-System and senior citizens in the SPR Project, is modelled with the use of emphasis as prosodic marker, placed on the appropriate elements of the utterance for achieving user-friendliness in man-machine communication in the sense of "accuracy" and "directness" (Hausser 2006) as well as (B) a sense of control to the user for the CSh System (Alexandris 2007, Nottas et al 2007 and (C) familiarity for the SPR System as an additional feature, according to the SPR Project's User Requirements. The evaluation of the prosodic markers in the SPR Project was primarily based on Interaction Aspects used for testing the performance of the CSh's System, namely (1) the Utterance, (2) Functional and (3) Satisfaction Levels (Moeller 2005) and also due both to the complexity of the CSh's System (a) user-group, a remarkable subset of which constitutes senior citizens, according to the CSh's Project User Requirements (Work-Package 1) and the (b) relatively positive results of the evaluation of the systems prosodic modelling in the Evaluation Phase (Work-Package 7) (of a 50-Test Users Group, 60% Men and 40% Women, Average Age: 35 -45, 80%, evaluated the Interface as "Very natural-sounding", and 50% evaluated the System as "Seems to inspire very high credibility").…”

“…In the first case, the semantic interpretation of the entire phrase or sentence may be determined by the type of element receiving prosodic emphasis, but the semantic content of the emphasized element itself is not effected. Based on the empirical data obtained both from the recorded corpora (ASR Component) and the evaluation of the spoken utterances produced by the Conversational Agent (Nottas et al 2007) in the CSh and SPR Projects we attempt to go a step further and provide (1) an integration of the results of previous studies (Alexandris et al 2005, Alexandris 2007 and to proceed to a (2) differentiation between specific word categories in relation to prosodic emphasis and their semantic content.…”

Controlling Interaction in Multilingual Conversation

“…The present approach is based on previous practices concerning the use of templates interacting within a Directed Dialog framework. A typical Dialog System involving the use of templates is the CitizenShield Dialog System for consumer complaints [3] [11] handling routine tasks involving food and manufactured products (namely complaints involving quality, product labels, prices etc.). The spoken input is automatically entered into templates containing a number of fields related to the categories and types of information concerning the product involved.…”