Evaluating Scientific Research in the Context of Prior Belief: Hindsight Bias or Confirmation Bias?

Abstract

When there is a mismatch between new evidence and prior beliefs, do people reject the conclusions from this evidence because of confirmation bias or do they support them because of hindsight bias? Ninety-four participants expressed a belief about a study's outcome before reading a research report. When belief was confirmed, the study's methodology was subsequently rated more positively and findings (whether presented with or without an explanation) were rated as more obvious, important, and interesting than when beliefs were disconfirmed. However, the presence of an explanation for the reported findings affected ratings of obviousness and interestingness but not of methodology. These results indicate that judging a research finding to be obvious involves more than a simple hindsight bias.

Influence Of Seductive Details, Belief-Congruence, And Repeated Testing On Memory For Controversial Information

“…What Masnick and Zimmerman (2009) found was that when the initial beliefs of participants were confirmed, they rated the methods of the experiment more positively;…”

“…Generally, many people seem to experience some kind of inherent cognitive difficulty in evaluating new information in an unbiased manner (Anderson, Lepper, & Ross, 1980;Bastardi, Uhlman, & Ross, 2011;Boysen & Vogel, 2007;Boysen & Vogel 2008;Corner et al, 2012;Levine & Murphy, 1943;Lord et al, 1984;Lord et al, 1979Lord et al, 1979Masnick & Zimmerman, 2009;Munro, Leary, & Lasane, 2004;Munro 2010;Plous, 1991;Reich, Green, Brock, & Tetlock, 2006;Wilson, DePaulo, Mook, & Klaaren, 1993), and may in fact work unknowingly to distort evidence that conflicts with their beliefs, or misremember what their beliefs were prior to encountering conflicting evidence as a means to better align evidence and belief Kuhn, 1989;Munro, Leary, & Lasane, 2004), even when explicitly instructed to attempt to be unbiased (Lord et al, 1984). Some of the factors that influence information processing include personal biases due to current beliefs (Boysen & Vogel, 2007;Boysen & Vogel, 2008;Corner et al ., 2012;Levine & Robinson, 1943;Lord, et al, 1984;Lord et al, 1979;Munro et al, 2004;Plous, 1991), perceived importance (Reich et al, 2006;Wilson et al, 1993), personal motivation (Bastardi, Uhlman, & Ross, 2011), and just being asked to form an opinion on a topic (Anderson, Lepper, & Ross 1980;Masnick & Zimmerman, 2009).…”

“…It has often been shown that when new information is consistent with an individual's current beliefs, biases, and desires, it will be rated more positively compared to when it is not (Bastardi Uhlman, & Ross, 2011;Corner et al, 2012;Lord et al, 1979;Munro et al, 2004). This can mean people will rate it as more obvious, more valid, more publishable, or having been attained through better methods (Bastardi, Uhlman, & Ross, 2011;Corner et al, 2012;Lord et al, 1979;Masnick & Zimmerman, 2009;Reich et al, 2006;Wilson et al, 1993).…”

“…In the experiments conducted by Lord et al (1979) and Bastardi et al (2011), the topics used for evaluation were presumably of high importance to the participants that evaluated the fictitious studies that were used. However, Anderson et al (1980) and Masnick and Zimmerman (2009) have both shown that it is not just topics for which people already have strong biases, desires, or long held beliefs about that they may evaluate in a biased manner. In Anderson et al (1980), the researchers sought to find whether participants would continue to support a theory about a topic they likely had not previously contemplated, even if the minimal evidence they were given for the theory was discredited; they also looked at whether making people provide their own explanations for the relation between the two variables being examined by a study claiming to lend support to the theory would increase their tendency to hold on to the theory after it was discredited.…”

“…In a similar line of research, Masnick and Zimmerman (2009) also looked at how people process information on a topic that is less politicized, and presumably one that people have less ingrained or solidified opinions about compared to those typically used in this line of research. In their study the topic was whether direct instruction was a better method of instruction in early science education compared to discovery learning.…”