The Use and Misuse of Classical Statistics: A Primer for Social Workers

Abstract

In this article, we offer a primer on “classical” frequentist statistics. In doing so, we aim to (1) provide social workers with a nuanced overview of common statistical concepts and tools, (2) clarify ways in which these ideas have oft been misused or misinterpreted in research and practice, and (3) help social workers better understand what frequentist statistics can and cannot offer. We begin broadly, starting with foundational issues in the philosophy of statistics. Then, we outline the Fisherian and Neyman–Pearson approaches to statistical inference and the practice of null hypothesis significance testing. We then discuss key statistical concepts including α, power, p values, effect sizes, and confidence intervals, exploring several common misconceptions about their use and interpretation. We close by considering some limitations of frequentist statistics and by offering an opinionated discussion on how social workers may promote more fruitful, responsible, and thoughtful statistical practice.

On the Dearth of Retractions in Social Work: A Preliminary Analysis of Ten Leading Journals

“…See Francis and Thunell (2022) and Heathers (2021) for an overview of these and other tools and techniques. For a discussion of how social work research may be made more reproducible/replicable, see Dunleavy and Lacasse (2023;especially Table 3).…”

“…Second, even when results are sufficiently described, a lack of open data precludes any possibility for meaningfully assessing computational reproducibility in quantitative studies or determining validity of findings in qualitative studies. 9 This, combined with a general disinterest (by social worker) in reproducibility and replicability (see discussion by Dunleavy & Lacasse, 2023) seem to have led the field to a place where every scholar wants to publish, but no one (readers, peer reviewers, editors) wants to meaningfully validate.…”

“…While the published literature is in some ways validated by this model, the underlying processes are largely unstandardized and opaque, and its overall functioning is poorly understood (see Dunleavy, 2022b;Tennant & Ross-Hellauer, 2019). Blatant errors (e.g., misreported or incorrect statistical results; inaccurate or misleading citations), omissions (e.g., selective reporting of results), misrepresentations (inflation or deflation of findings), and even cases of fraud (e.g., fabrication of data, falsification) are inevitably published (see Dunleavy & Lacasse, 2023;Ioannidis, 2005Ioannidis, , 2008Nuijten et al, 2015;Srivastava, 2016, and references therein).…”