Can $p$-values be meaningfully interpreted without random sampling?

Hirschauer, Norbert; Grüner, Sven; Mußhoff, Oliver; Becker, Claudia; Jantsch, Antje

doi:10.1214/20-ss129

Cited by 37 publications

(32 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non‐compliance, in contrast, implies that all sample quantities remain pure data descriptions. They are devoid of inferential meaning (except when deviations from random sampling are adequately corrected within a sample selection model) 2 . More generally speaking, all statistical inferential procedures based on the standard error – including statistical significance tests – are inappropriate for making sample‐to‐population inferences when studies are based on non‐random samples.…”

Section: Statistics Are Sample Quantitiesmentioning

confidence: 99%

See 1 more Smart Citation

Inference Using Non-Random Samples? Stop Right There!

Hirschauer

Grüner

Mußhoff³

et al. 2021

Significance

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Statistics Are Sample Quantitiesmentioning

confidence: 99%

“…This short article is based on the paper “Can p ‐values be meaningfully interpreted without random sampling?”, published in Statistics Surveys 2 …”

Section: Notementioning

confidence: 99%

Inference Using Non-Random Samples? Stop Right There!

Hirschauer

Grüner

Mußhoff³

et al. 2021

Significance

Self Cite

View full text Add to dashboard Cite

show abstract

“…Despite many such clear guidelines from statisticians and methodologists, NHST is still used routinely on nonrandom samples and on populations, where inference is inappropriate (see Berk et al, 1995;Hirschauer et al, 2019;Hirschauer et al, 2020;Potocky-Tripodi & Tripodi, 2003, for discussion). R. A.…”

Section: Objections To Null Hypothesis Testingmentioning

confidence: 99%

The Use and Misuse of Classical Statistics: A Primer for Social Workers

Dunleavy

Lacasse

2021

Research on Social Work Practice

View full text Add to dashboard Cite

In this article, we offer a primer on “classical” frequentist statistics. In doing so, we aim to (1) provide social workers with a nuanced overview of common statistical concepts and tools, (2) clarify ways in which these ideas have oft been misused or misinterpreted in research and practice, and (3) help social workers better understand what frequentist statistics can and cannot offer. We begin broadly, starting with foundational issues in the philosophy of statistics. Then, we outline the Fisherian and Neyman–Pearson approaches to statistical inference and the practice of null hypothesis significance testing. We then discuss key statistical concepts including α, power, p values, effect sizes, and confidence intervals, exploring several common misconceptions about their use and interpretation. We close by considering some limitations of frequentist statistics and by offering an opinionated discussion on how social workers may promote more fruitful, responsible, and thoughtful statistical practice.

show abstract

“…There are so many such assumptions that Bradley andBrand (2016) andTrafimow (2019) proposed assumption taxonomies. For instance, a typical assumption is that the researcher has sampled randomly and independently from the population (Berk and Freeman, 2003;Hirschauer, 2020), an assumption that has never been true in the fields of psychology, marketing, or management. Let H denote the test (null) hypothesis and let A denote the large set of additional assumptions that go into a statistical model M: thus, M = H + A.…”

Section: The Issue Of Hypotheses Versus Modelsmentioning

confidence: 99%

Distinguishing Between Models and Hypotheses: Implications for Significance Testing

Trafimow¹

2021

Preprint

View full text Add to dashboard Cite

In the debate about the merits or demerits of null hypothesis significance testing (NHST), authorities on both sides assume that the p value that a researcher computes is based on the null hypothesis or test hypothesis. If the assumption is true, it suggests that there are proper uses for NHST, such as distinguishing between competing directional hypotheses. And once it is admitted that there are proper uses for NHST, it makes sense to educate substantive researchers about how to use NHST properly and avoid using it improperly. From this perspective, the conclusion would be that researchers in the business and social sciences could benefit from better education pertaining to NHST. In contrast, my goal is to demonstrate that the p value that a researcher computes is not based on a hypothesis, but on a model in which the hypothesis is embedded. In turn, the distinction between hypotheses and models indicates that NHST cannot soundly be used to distinguish between competing directional hypotheses or to draw any conclusions about directional hypotheses whatsoever. Therefore, it is not clear that better education is likely to prove satisfactory. It is the temptation issue, not the education issue, that deserves to be in the forefront of NHST discussions.

show abstract

Can $p$-values be meaningfully interpreted without random sampling?

Cited by 37 publications

References 49 publications

Inference Using Non-Random Samples? Stop Right There!

Inference Using Non-Random Samples? Stop Right There!

The Use and Misuse of Classical Statistics: A Primer for Social Workers

Distinguishing Between Models and Hypotheses: Implications for Significance Testing

Contact Info

Product

Resources

About