Pragmatics in human-computer conversations

Abstract

This paper provides a pragmatic analysis of some human-computer conversations carried out during the past six years within the context of the Loebner Prize Contest, an annual competition in which computers participate in Turing Tests. The Turing Test posits that to be granted intelligence, a computer should imitate human conversational behavior so well as to be indistinguishable from a real human being. We carried out an empirical study exploring the relationship between computers' violations of Grice's cooperative principle and conversational maxims, and their success in imitating human language use. Based on conversation analysis and a large survey, we found that different maxims have different effects when violated, but more often than not, when computers violate the maxims, they reveal their identity. The results indicate that Grice's cooperative principle is at work during conversations with computers. On the other hand, studying human-computer communication may require some modifications of existing frameworks in pragmatics because of certain characteristics of these conversational environments. Pragmatics constitutes a serious challenge to computational linguistics. While existing programs have other significant shortcomings, it may be that the biggest hurdle in developing computer programs which can successfully carry out conversations will be modeling the ability to 'cooperate'. 0 2002 Elsevier Science B.V. All rights reserved.

Artificial conversations for customer service chatter bots: Architecture, algorithms, and evaluation metrics

“…Contemporary chatter bots perform very effectively in question-answer settings and similar utterance-exchange pair settings, where the contexts of the conversations are independent from one exchange to the next (Mauldin, 1994;Saygin & Ciceklib, 2002;Chakrabarti & Luger, 2012). However, they perform poorly in conversational situations where a specific context is maintained through a series of several utterance-exchange pairs.…”

“…⁄ relation: speaker's utterance is relevant to the context and the topic of the conversation ⁄ manner: speaker's utterance is direct and straightforward, without ambiguity or obfuscation Saygin and Ciceklib (2002) demonstrated that evaluating chatter bots using Grice's cooperative maxims is an effective way to compare chatter bots competing for the Loebner prize. They provide a scoring matrix, against which each artificial conversation can be graded for a specific criterion and hence offer a good potential starting point for evaluating our artificial conversations.…”

Artificial Intelligence Now and the Future

“…So researchers in the field of AI have the challenge of designing a machine, endowed with functions at birth (at design), and improves with growth like a child, and learns externally (from its environment) without any further capabilities bestowed (internally) on it by its designers. An interesting statement is made by Singh and Gupta [25]: "A true AI program should be able to perceive the world around it, autonomously decide its actions and should be able to adapt itself to the changes." This reflects the requirements of a machine to pass Schweizer"s [26] Truly Total Turing Test (TTTT).…”

Learning a Super Mario controller from examples of human play

“…3) Design: To ensure the evaluation was correct, we set up the questionnaires in the following steps which can help us organize the clear comparison between human and computer performance, as in the modified Turing Test for dialogue provided in [15]:…”