Multiple Perspectives on Inference for Two Simple Statistical Scenarios

Dongen, Noah N. N. van; Doorn, Johnny van; Gronau, Quentin Frederik; Ravenzwaaij, Don van; Hoekstra, Rink; Haucke, Matthias; Lakens, Daniël; Hennig, Christian; Morey, Richard D.; Homer, Saskia; Gelman, Andrew; Sprenger, Jan; Wagenmakers, Eric Jan

doi:10.1080/00031305.2019.1565553

Cited by 37 publications

(29 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent study in the gerontology literature in which four null effects were evaluated with equivalence tests or Bayes factors ( Lakens, McLatchie, Isager, Scheel, & Dienes, 2020 ), both approaches led to similar inferences in each example. Likewise, four teams of researchers using frequentist or Bayesian hypothesis testing or estimation independently reached similar conclusions when reanalyzing two studies ( Dongen et al, 2019 ). Although one can always find exceptions if one searches long enough, in most cases Bayes factors and p values will strongly agree ( Tendeiro & Kiers, 2019 ).…”

Section: Why Would Alternatives To P Values Fare Any Better?mentioning

confidence: 77%

The Practical Alternative to the p Value Is the Correctly Used p Value

Lakens

2021

Perspect Psychol Sci

View full text Add to dashboard Cite

Because of the strong overreliance on p values in the scientific literature, some researchers have argued that we need to move beyond p values and embrace practical alternatives. When proposing alternatives to p values statisticians often commit the “statistician’s fallacy,” whereby they declare which statistic researchers really “want to know.” Instead of telling researchers what they want to know, statisticians should teach researchers which questions they can ask. In some situations, the answer to the question they are most interested in will be the p value. As long as null-hypothesis tests have been criticized, researchers have suggested including minimum-effect tests and equivalence tests in our statistical toolbox, and these tests have the potential to greatly improve the questions researchers ask. If anyone believes p values affect the quality of scientific research, preventing the misinterpretation of p values by developing better evidence-based education and user-centered statistical software should be a top priority. Polarized discussions about which statistic scientists should use has distracted us from examining more important questions, such as asking researchers what they want to know when they conduct scientific research. Before we can improve our statistical inferences, we need to improve our statistical questions.

show abstract

Section: Why Would Alternatives To P Values Fare Any Better?mentioning

confidence: 77%

The Practical Alternative to the p Value Is the Correctly Used p Value

Lakens

2021

Perspect Psychol Sci

View full text Add to dashboard Cite

show abstract

“…Within hypothesis generating research we primarily reveal what might be important, while within hypothesis testing research we focus on specifically excluding competing hypotheses. Within hypothesis generating research, since we are often exploring what the data reveals, it can be useful to consider and contrast the outcome from different methodologies to see what is found consistently to be significant (25). However, researchers will often only report the outcome of a single test.…”

Section: Introductionmentioning

confidence: 99%

Potential Risk Factors for Aggression and Playfulness in Cats: Examination of a Pooling Fallacy Using Fe-BARQ as an Example

et al. 2021

View full text Add to dashboard Cite

Using a popular method of behaviour evaluation which rates the intensity of behaviour in different contexts, we demonstrate how pooling item scores relating to a given construct can reveal different potential risk factors for the dependent variable depending on how the total score is constructed. We highlight how similar simple total scores can be constructed through very different combinations of constituent items. We argue for the importance of examining individual item score distributions, and the results from different intensity thresholds before deciding on the preferred method for calculating a meaningful dependent variable. We consider simply pooling individual item scores which conflate context with intensity to calculate an average score and assuming this represents a biologically meaningful measure of trait intensity is a fallacy. Specifically using four items that describe intercat aggression and eleven that describe playfulness in cats in Fe-BARQ, we found sex and neuter status, social play and fearfulness were consistently significant predictors for intercat aggression scores; and age, age when obtained, social play and fearfulness were significant predictors of playfulness scores. However, the significance of other factors such as scratching varied with the threshold used to calculate to the total score. We argue that some of these inconsistent variables may be biologically and clinically important and should not be considered random error. Instead they need to be evaluated in the context of other available evidence.

show abstract

“…Evidence from P values and Bayes factors can be interpreted jointly, 16 and may point in the same direction, 17 so we are advocating for the increased consideration of the Bayesian framework in our statistical toolbox.…”

Section: The Bayesian Framework Explainedmentioning

confidence: 99%