Effect of Confidence Interval Construction on Judgment Accuracy

Mandel, David R.; Collins, Robert N.; Ef, Risko; Ja, Fugelsang

doi:10.31234/osf.io/mktgj

Cited by 8 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AOT assesses an individual's willingness to evaluate evidence contrary to their beliefs as well as openness to alternative perspectives (Baron et al, 2015). AOT is positively related to accuracy in a variety of judgment tasks (Haran et al, 2013;Mandel et al, 2020) and negatively related to some cognitive biases (Toplak et al, 2017). 1 Finally, in addition to examining the relation between the BBS and these measures of cognitive ability, we also examine the relation between the individual self and other ratings, which has not been explored in earlier studies.…”

Section: The Cognitive Sophistication Hypothesismentioning

confidence: 99%

Hypothesized Drivers of the Bias Blind Spot—Cognitive Sophistication, Introspection Bias, and Conversational Processes

Mandel¹,

Collins²,

Walker³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Individuals often assess themselves as being less susceptible to common biases compared to others. This bias blind spot is thought to represent a metacognitive error with potentially profound consequences. In this research, we tested three explanations for the effect: The metacognitive insight hypothesis posits that individuals who display the bias blind spot more strongly are actually less biased than others. The introspection bias hypothesis posits that the bias blind spot occurs because people rely on introspection more when assessing themselves compared to others. The conversational processes hypothesis posits that the effect is largely a consequence of the pragmatic aspects of the experimental situation. In two well-powered experiments (N = 1057) examining 18 social/motivational and cognitive biases, there was strong evidence of bias blind spots. Among the three hypotheses examined, the conversational hypothesis attracted the greatest support, thus raising questions about the extent to which the bias blind spot is a metacognitive effect.

show abstract

Section: The Cognitive Sophistication Hypothesismentioning

confidence: 99%

Hypothesized Drivers of the Bias Blind Spot—Cognitive Sophistication, Introspection Bias, and Conversational Processes

Mandel¹,

Collins²,

Walker³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This means that findings from studies where best point and interval judgments were made one after another within the same task session, e.g. [ 30 ], may not generalize to situations where only one way of assessment is used. Furthermore, having people specify where in the interval they think the best guess is with the splitting method (which is similar to defining a point in the interval), may not necessarily lead towards an estimate that conceptually corresponds to a point judgment.…”

Section: Discussionmentioning

confidence: 99%

Self-selected interval judgments compared to point judgments: A weight judgment experiment in the presence of the size-weight illusion

et al. 2022

View full text Add to dashboard Cite

Measurements of human attitudes and perceptions have traditionally used numerical point judgments. In the present study, we compared conventional point estimates of weight with an interval judgment method. Participants were allowed to make step by step judgments, successively converging towards their best estimate. Participants estimated, in grams, the weight of differently sized boxes, estimates thus susceptible to the size-weight illusion. The illusion makes the smaller of two objects of the same weight, differing only in size, to be perceived as heavier. The self-selected interval method entails participants judging a highest and lowest reasonable value for the true weight. This is followed by a splitting procedure, consecutive choices of selecting the upper or lower half of the interval the individual estimates most likely to include the true value. Compared to point estimates, interval midpoints showed less variability and reduced the size-weight illusion, but only to a limited extent. Accuracy improvements from the interval method were limited, but the between participant variation suggests that the method has merit.

show abstract

“…This technique [23,24] is thought to encourage information sampling [23] and prompt participants to consider counter-arguments [25]. (However, see also [26] for counter-evidence regarding this sequencing. )…”

Section: Research Goalsmentioning

confidence: 99%

Predicting reliability through structured expert elicitation with repliCATS (Collaborative Assessments for Trustworthy Science)

Fraser¹,

Bush²,

Wintle³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Replication is a hallmark of scientific research. As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce a new technique to evaluating replicability, the repliCATS (Collaborative Assessments for Trustworthy Science) process, a structured expert elicitation approach based on the IDEA protocol. The repliCATS process is delivered through an underpinning online platform and applied to the evaluation of research claims in social and behavioural sciences. This process can be deployed for both rapid assessment of small numbers of claims, and assessment of high volumes of claims over an extended period. Pilot data suggests that the accuracy of the repliCATS process meets or exceeds that of other techniques used to predict replicability. An important advantage of the repliCATS process is that it collects qualitative data that has the potential to assist with problems like understanding the limits of generalizability of scientific claims. The repliCATS process has potential applications in alternative peer review and in the allocation of effort for replication studies.

show abstract

Effect of Confidence Interval Construction on Judgment Accuracy

Cited by 8 publications

References 45 publications

Hypothesized Drivers of the Bias Blind Spot—Cognitive Sophistication, Introspection Bias, and Conversational Processes

Hypothesized Drivers of the Bias Blind Spot—Cognitive Sophistication, Introspection Bias, and Conversational Processes

Self-selected interval judgments compared to point judgments: A weight judgment experiment in the presence of the size-weight illusion

Predicting reliability through structured expert elicitation with repliCATS (Collaborative Assessments for Trustworthy Science)

Contact Info

Product

Resources

About