Bias in the evaluation of psychology studies: A comparison of parapsychology versus neuroscience

Abstract

Research suggests that scientists display confirmation biases with regard to the evaluation of research studies, in that they evaluate results as being stronger when a study confirms their prior expectations. These biases may influence the peer review process, particularly for studies that present controversial findings. The purpose of the current study was to compare academic psychologists' evaluation of a parapsychology study versus a neuroscience study. One hundred participants with a background in psychology were randomly assigned to read and evaluate one of two virtually identical study abstracts (50 participants per group). One of the abstracts described the findings as if they were from a parapsychology study, whereas the other abstract described the findings as if they were from a neuroscience study. The results revealed that participants rated the neuroscience abstract as having stronger findings and as being more valid and reliable than the parapsychology abstract, despite the fact that the two abstracts were identical. Participants also displayed confirmation bias in their ratings of the parapsychology abstract, in that their ratings were correlated with their scores on transcendentalism (a measure of beliefs and experiences related to parapsychology, consciousness and reality). Specifically, higher transcendentalism was associated with more favorable ratings of the parapsychology abstract, whereas lower transcendentalism was associated with less favorable ratings. The findings suggest that psychologists need to be vigilant about potential biases that could impact their evaluations of parapsychology research during the peer review process.

Does preregistration improve the credibility of research findings?

“…Confirmation bias is a tendency for researchers to interpret new evidence as confirming their hypotheses. This bias is stronger when researchers view hypotheses as being more plausible (e.g., Butzer, 2019;Hergovich, Schott, & Burger, 2010). Hence, confirmation bias facilitates scientific progress by preventing well-established, highly plausible theories from being disconfirmed too easily.…”

Investigations into retrocausal effects in cueing tasks.

“…While parapsychology may be controversial within mainstream academia (see Butzer, 2020, for an example), it is worth noting that here, in particular the first author considers himself skeptical regarding parapsychological effects, and we have made all attempts to make the experiments used as “standard” as possible, with minimal changes between the standard and retrocausal conditions. The skeptic could argue this is why we did not find significant effects in five of seven experiments.…”

UnityGPT vs. ChatGPT: A Comparative Review for Frontier Scientists

“…Given its foundational training data, the model will naturally be inclined towards the prevailing scientific orthodoxy and the material reductionist philosophy of the times. Many researchers, either implicitly or overtly, adopt this framework, which is inevitably reflected in the language in the works they produced (Butzer, 2020). Notably, discourse surrounding parapsychology often harbors a dismissive or patronizing undertone (See: Panel 1 of Appendix Fig.…”