Field trial evaluations of two different information inquiry systems

Abstract

This paper presents the evaluations of the field trials of two information inquiry systems that use different speech technologies and different human-machine interfaces. The first system, RAILTEL, uses isolated word recognition and system driven dialogue. The second system, Dialogos, understands spontaneous speech and implements a mixed initiative dialogue strategy. Both the systems allow to access the Italian Railway timetable by using the telephone over the public network. Both the systems were tested in extensive field trials by inexperienced subjects. Moreover a comparative trial with a limited number of subjects was performed in order to get some insights on the impact of the use of different speech technologies on the user behaviour. After a brief description of the two systems, the paper will report the details of the experimental set up of the trials and the discussion of the results.

Dialogue strategies for improving the usability of telephone human-machine communication

“…The subjects of this experiment called Dialogos and another spoken dialogue system. Each of them played two different predetermined scenarios with each system; after the trial, they were interviewed (the discussion of this experiment may be found in (Billi, Castagneri, and Danieli, 1997)). By analyzing the 96 dialogues between Dialogos and the experimental subjects, we found that the occurrence of users' errors could be classified into the two classes described above, that is users' errors were always condomitant with substitutions or insertions in the best-decoded sequence.…”

Speech Standards: Lessons Learnt

“…• Spoken dialog systems (SDS) research was initially promoted by EU-funded projects, such as SUNDIAL [7], RAILTEL [8], and its continuation ARISE [9]. The results achieved in those projects were very promising to the point that the Italian Railways company (Ferrovie dello Stato, now Trenitalia) decided to deploy the prototype developed within the ARISE project with the help of Telecom Italia Labs (TILAB).…”

Procesamiento de Informacion Multilingüe en un Sistema de Diálogo

“…Los sistemas de diálogo son programas de computador diseñados para interactuar oralmente con los usuarios con objeto de proporcionar determinados servicios, como por ejemplo, información y reserva de viajes en avión (Seneff y Polifroni, 2000), información meteorológica (Zue et al 2000;Padrell y Hernando, 2002;Wang et al 2000;Nakano et al 2001), compra de productos (López-Cózar et al 2002a), información y reserva de viajes en tren (Billi et al 1997), etc. Dado que el habla constituye la forma más natural de comunicación entre las personas, la finalidad de estos sistemas es aumentar la rapidez, efectividad y facilidad a la hora de realizar este tipo de tareas de forma automática.…”