Language in dialogue: when confederates might be hazardous to your data

Abstract

Experiments that aim to model language processing in spoken dialogue contexts often use confederates as speakers or addressees. However, the decision of whether to use a confederate, and of precisely how to deploy one, is shaped by researchers' explicit theories and implicit assumptions about the nature of dialogue. When can a confederate fulfill the role of conversational partner without changing the nature of the dialogue itself? We survey the benefits and risks of using confederates in studies of language in dialogue contexts, identifying four concerns that appear to guide how confederates are deployed. We then discuss several studies that have addressed these concerns differently-and, in some cases, have found different results. We conclude with recommendations for how to weigh the benefits and risks of using experimental confederates in dialogue studies: Confederates are best used when an experimental hypothesis concerns responses to unusual behaviors or low-frequency linguistic forms and when the experimental task calls for the confederate partner to take the initiative as speaker. Confederates can be especially risky in the addressee role, especially if their nonverbal behavior is uncontrolled and if they know more than is warranted by the experimental task.

Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape

“…Recently, there has been an increasing focus on reproducibility in psychology (Open Science Collaboration, 2015) and worries about claims that only certain researchers have the right ‘flair’ to replicate studies (Baumeister, 2016). Confederate studies in particular may be susceptible to such effects (Doyen, Klein, Pichon, & Cleeremans, 2012), or participants may be behave differently with confederates (Kuhlen & Brennan, 2013). All these factors make confederate interaction studies hard to replicate.…”

On what happens in gesture when communication is unsuccessful

“…Kuhlen and Brennan, 2013). This could involve, for example, communication about Greebles as well, in which miscommunications (of various kinds) may occur in a more natural way.…”

An Appeal for a Methodological Fusion of Conversation Analysis and Experimental Psychology

“…Also, in terms of what is interactionally relevant in any given situation, there is no intrinsic distinction between "natural" and "contrived" data (Speer, 2002). We therefore propose combining validation studies that explore the limits and consequences of artificial scenarios, role-playing, and the use of confederates (see, e.g., Kuhlen & Brennan, 2013) with CA studies of experimental scenarios as interactional occasions. For example, CA studies of experiments based on introspective self-report have provided valuable insight into the pragmatics and accountability of introspection (Wooffitt & Holt, 2011).…”