The citation bias: Fad and fashion in the judgment and decision literature.

Christensen-Szalanski, Jay; Beach, Lee Roy

doi:10.1037/0003-066x.39.1.75

Cited by 165 publications

(70 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We would hope, and some studies have shown (Christensen -Szalanski and Beach, 1984 ), that individuals are better calibrated on subjects they know about. The concentrations of benzene in ambient, indoor, and personal air have been studied extensively in the last 10 to 15 years and are better characterized than the levels of most chemicals.…”

Section: Discussionmentioning

confidence: 99%

Use of expert judgment in exposure assessment: Part 2. Calibration of expert judgments about personal exposures to benzene

Walker

Catalano

Hammitt

et al. 2003

J Expo Sci Environ Epidemiol

View full text Add to dashboard Cite

The recent movement of regulatory agencies toward probabilistic analyses of human health and environmental risks has focused greater attention on the quality of the estimates of variability and uncertainty that underlie them. Of particular concern is how uncertainty -a measure of what is not known -is characterized, as uncertainty can play an influential role in analyses of the need for regulatory controls or in estimates of the economic value of additional research. This paper reports the second phase of a study, conducted as an element of the National Human Exposure Assessment Survey ( NHEXAS ), to obtain and calibrate exposure assessment experts' judgments about uncertainty in residential ambient, residential indoor, and personal air benzene concentrations experienced by the nonsmoking, nonoccupationally exposed population in U.S. EPA's Region V. Subjective judgments ( i.e., the median, interquartile range, and 90% confidence interval ) about the means and 90th percentiles of each of the benzene distributions were elicited from the seven experts participating in the study. The calibration or quality of the experts' judgments was assessed by comparing them to the actual measurements from the NHEXAS Region V study using graphical techniques, a quadratic scoring rule, and surprise and interquartile indices. The results from both quantitative scoring methods suggested that, considered collectively, the experts' judgments were relatively well calibrated although on balance, underconfident. The calibration of individual expert judgments appeared variable, highlighting potential pitfalls in reliance on individual experts. In a surprising finding, the experts' judgments about the 90th percentiles of the benzene distributions were better calibrated than their predictions about the means; the experts tended to be overconfident in their ability to predict the means. This paper is also one of the first calibration studies to demonstrate the importance of taking into account intraexpert correlation on the statistical significance of the findings. When the judgments were assumed to be independent, analysis of the surprise and interquartile indices found evidence of poor calibration ( P < 0.05 ). However, when the intraexpert correlation in the study was taken into account, these findings were no longer statistically significant. The analysis further found that the experts' judgments scored better than estimates of Region V benzene concentrations simply drawn from earlier studies of ambient, indoor and personal benzene levels in other U.S. cities. These results suggest the value of careful elicitation of expert judgments in characterizing exposures in probabilistic form. Additional calibration studies need to be undertaken to corroborate and extend these findings.

show abstract

Section: Discussionmentioning

confidence: 99%

Use of expert judgment in exposure assessment: Part 2. Calibration of expert judgments about personal exposures to benzene

Walker

Catalano

Hammitt

et al. 2003

J Expo Sci Environ Epidemiol

View full text Add to dashboard Cite

show abstract

“…In commenting on "the rhetoric of irrationality," Lopes (1991) noted an overemphasis in the judgment and decision-making literature on the negative aspects of people's performance and a comparative neglect of situations in which they reason well (see also Christensen-Szalanski & Beach, 1984). What appears to be a bias in the laboratory may be functional behavior in a more realistic context (Funder, 1987;Hogarth, 1981), where a variety of justifications for the behavior can be considered.…”

Section: Discussionmentioning

confidence: 99%

Learning lessons from sunk costs.

Bornstein¹,

Chapman²

1995

Journal of Experimental Psychology: Applied

View full text Add to dashboard Cite

Study participants rated the quality of several arguments for continuing an original plan in sunk cost situations in order to (a) avoid wasting resources, (b) learn to make better decisions, (c) punish poor decision making, and (d) appear consistent. The lesson-learning argument was perceived as most appropriate when adult teachers taught lessons to others, the original decision was carelessly made, or if it consumed comparatively more resources. Ratings of the lesson-learning argument were higher for teacher–learner than for adult-alone situations, regardless of whether the learner was a child or an adult. The implications for improving decision making and judging whether the sunk cost effect is a bias are discussed.

show abstract

“…Studies with larger effects may be more likely to be cited than studies with small, nonsignificant effects (Christensen-Szalanski & Beach, 1984;Jannot, Agoritsas, Gayet-Ageron, & Perneger, 2013). Citation bias can cause effects to look more important or undisputed than they really are when taking into consideration all relevant evidence.…”

Section: Citation Biasmentioning

confidence: 99%

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

SUMMARY DOCTORAL DISSERTATION: Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes. We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.

show abstract

The citation bias: Fad and fashion in the judgment and decision literature.

Cited by 165 publications

References 30 publications

Use of expert judgment in exposure assessment: Part 2. Calibration of expert judgments about personal exposures to benzene

Use of expert judgment in exposure assessment: Part 2. Calibration of expert judgments about personal exposures to benzene

Learning lessons from sunk costs.

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Contact Info

Product

Resources

About