Dealing with the Conflicting Results of Psycholinguistic Experiments: How to Resolve Them with the Help of Statistical Meta-analysis

Abstract

This paper proposes the use of the tools of statistical meta-analysis as a method of conflict resolution with respect to experiments in cognitive linguistics. With the help of statistical meta-analysis, the effect size of similar experiments can be compared, a well-founded and robust synthesis of the experimental data can be achieved, and possible causes of any divergence(s) in the outcomes can be revealed. This application of statistical meta-analysis offers a novel method of how diverging evidence can be dealt with. The workability of this idea is exemplified by a case study dealing with a series of experiments conducted as non-exact replications of Thibodeau and Boroditsky (PLoS ONE 6(2):e16782, 2011. https://doi.org/10.1371/journal.pone.0016782 ).

Systematic literature reviews: Four applications for interdisciplinary research

“…For instance, meta-analyses are a popular method of research synthesis in applied linguistics (Ellis, 2015;Norris & Ortega, 2007). By contrast, meta-analysis seems less well known in psycholinguistics, given that a recent paper introduced meta-analysis as a "novel method" to the field (Rákosi, 2018). Across academic disciplines, different meta-analyses have looked at the persuasive impact of verbal metaphors (e.g., Rákosi, 2018;Sopory & Dillard, 2002, Van Stee, 2018.…”

“…By contrast, meta-analysis seems less well known in psycholinguistics, given that a recent paper introduced meta-analysis as a "novel method" to the field (Rákosi, 2018). Across academic disciplines, different meta-analyses have looked at the persuasive impact of verbal metaphors (e.g., Rákosi, 2018;Sopory & Dillard, 2002, Van Stee, 2018. These meta-analyses consistently found that metaphors have a statistically small effect on persuasion, and that effect sizes differ between metaphors, topics, and other conditions.…”

“…Because many of these framing studies did not explicitly reference metaphor, they were excluded from earlier meta-analyses into the effects of metaphor. These earlier meta-analyses (e.g., Rákosi, 2018;Sopory & Dillard, 2002, Van Stee, 2018) mainly focused on metaphorical-words frames. Thus, we conducted a meta-analysis in which we contrasted effects of metaphorical-words frames and metaphorical-concept frames (Brugman, Burgers & Vis, in press).…”

Creative Argumentation: When and Why People Commit the Metaphoric Fallacy

“…In particular, we expected that the framing effect of metaphor might also depend on the type of metaphor featured in the first premise of the argument. In fact, conventional metaphors are not perceived as metaphors, bearing a status similar to polysemous literal term ( Carston, 2002 ; Giora, 2003 ; Bowdle and Gentner, 2005 ; Kenett et al, 2018 ), and unconsciously act as triggers of “systems of commonplaces” or background of associated beliefs ( Black, 1954 ; Coulson, 2001 ; Lakoff, 2004 ; Thibodeau and Boroditsky, 2011 , 2013 ). Novel metaphors, however, are consciously processed as metaphors and require new, original and creative interpretations ( Indurkhya, 2007b , 2010 ; Kenett et al, 2018 ).…”

“…The mapping of properties from the source to the target implicitly forces the interpreter to consider the target in a specific perspective. Therefore, different metaphorical views might seriously affect one’s reasoning and evaluation of arguments ( Lakoff, 2004 ; Goatly, 2007 ; Thibodeau and Boroditsky, 2011 , 2013 ; Steen et al, 2014 ; Semino et al, 2016 ).…”