Cooperation in Online Conversations: The Response Times as a Window Into the Cognition of Language Processing

Abstract

Measuring the cognitive cost of interpreting the meaning of sentences in a conversation is a complex task, but it is also at the core of Sperber and Wilson's Relevance Theory. In cognitive sciences, the delay between a stimulus and its response is often used as an approximation of the cognitive cost. We have noticed that such a tool had not yet been used to measure the cognitive cost of interpreting the meaning of sentences in a free-flowing and interactive conversation. The following experiment tests the ability to discriminate between sentences with a high cognitive cost and sentences with a low cognitive cost using the response time of the participants during an online conversation in a protocol inspired by the Turing Test. We have used violations of Grice's Cooperative Principle to create conditions in which sentences with a high cognitive cost would be produced. We hypothesized that response times are directly correlated to the cognitive cost required to generate implicatures from a statement. Our results are coherent with the literature in the field and shed some new light on the effect of violations on the humanness of a conversational agent. We show that violations of the maxim of Relation had a particularly important impact on response times and the perceived humanness of a conversation partner. Violations of the first maxim of Quantity and the fourth maxim of Manner had a lesser impact, and only on male participants.

Contextual Information Helps Understand Messages Written with Textisms

“…The experiment was adapted from Jacquet et al [32,33]. The main differences were that in these experiments, the authors investigated the influence of violations of the Gricean conversational maxims on the response times of participants in a Turing test [34], with the experimenter violating the maxims in one conversation and respecting them in the other.…”

“…Depending on the experiments, indirect measures can be made to estimate the cognitive cost required to process stimuli and to respond to them. Some of these indirect measures include recording pupil dilation (which requires specific instruments and heavy control to mitigate the influence of other factors) [30], the success rate in primary or secondary tasks [31], and latencies such as response times used in the present research (or reaction times for fast processes) [32,33].…”

“…Any interpretation should take this fact into account. Response times are easy to record in online conversations, and different studies have shown that they are compatible with how the cognitive cost should vary when studying violations of conversational expectations [32,33]. Jacquet et al [32,33] indeed showed that during a Turing test (i.e., a test evaluating how human a conversational agent feels by comparing two conversations, one with the tested agent and the other with a human reference [34,35]), the participants' response times were significantly higher when the experimenter violated Grice's maxim of relation and Grice's maxim of quantity (giving too much information) in a conversation, compared to a conversation in which the Gricean maxims were respected.…”

“…To do this, we used a linear regression model taking into account the length of the experimenter's message (L e ) and the length of the participant's reply (L p ) to predict the theoretical delay in normal conversations (D). Just as in [32,33], we used a simple linear model here to define the average reading and writing speed of participants, as the number of turns during a conversation was generally not enough to get a good model for each individual participant.…”

Toward a linguistically grounded dialog model for chatbot design

“…He can be considered one of the pioneers of inferential pragmatics (Ariel 2012). Most of all the cooperation principle and the conversational maxims proposed by Grice (1975) and later updated by Sperber and Wilson (1995) are a consistent part of the study of pragmatics today and involved in the implementation of dialogue systems from a methodological point of view: Jacquet et al (2018;2019b) evaluate the violation of the gricean maxims in textual online conversations; Saygin and Cicekli (2002) propose an empirical study study of human-computer interactions within the context of the Loebner Prize Contest.…”