Extraordinary Claims, Extraordinary Evidence? A Discussion

Abstract

Roberts (2020) discussed research claiming honeybees can do arithmetic. Some readers of this research might regard such claims as unlikely. The present authors used this example as a focus for a debate on the criterion that ought to be used for publication of results that could be viewed as unlikely by a significant number of readers. The resulting dialogue contains interesting discussion of non-human cognition, whether honeybee arithmetic should be considered unlikely, and the role of replication in such cases.

Doing reliable research in comparative psychology: Challenges and proposals for improvement.

“…We should reflect on what evidence is required for acceptance of a claim, and in particular, extraordinary claims (Shiffrin et al, 2021). For example, for particularly “surprising” or “exceptional” novel results (though how this would be determined, given that what constitutes “surprising” is a subjective judgment based on prior knowledge and biases is unclear), more extensive/convincing evidence might be required.…”

“…More extensive evidence might consist of study replication prior to publication (see, e.g., Leonard et al, 2017 for an example of this in infant cognition), or converging evidence from multiple validated measures. A tiered approach to publication that more accurately reflects the more nuanced way we think about evidence could facilitate this; for example, journals could have separate sections for the reporting of (a) results (i.e., technical reports) and (b) claims, where the results of several studies are synthesized (Shiffrin et al, 2021). For comparative psychology, a section for the reporting of anecdotal and serendipitous findings that are often only discussed informally with colleagues could be valuable, given the important role these play when studying animals (Burghardt, 2013; Kamil, 1987).…”