Reconsidering the Fundamental Contributions of Fisher and Neyman on Experimentation and Sampling

Fienberg, Stephen E.; Tanur, Judith M.

doi:10.2307/1403784

Cited by 41 publications

(26 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In reasoning from NHST, the commonly used definition of the P‐ value as ‘a measure of evidence against the null hypothesis’ is potentially misleading in that it seems to legitimise transposing the conditional as if it were a mathematically valid function rather than a matter of intuition. It was the intuitive interpretation that Fisher used in his a posteriori model of NHST . His aim was to use the P‐ value as an aid in deciding which experiments to repeat.…”

Section: What Is a P‐value And What Does It Measure?mentioning

confidence: 99%

“…It was the intuitive interpretation that Fisher used in his a posteriori model of NHST. 8,9 His aim was to use the P-value as an aid in deciding which experiments to repeat. If on several repetitions, a consistent extreme P-value for the sample statistic was obtained then that would accumulate evidence for a true experimental effect.…”

Section: What Is a P-value And What Does It Measure?mentioning

confidence: 99%

“…This erroneous interpretation has arisen from the illusion of coherence resulting from the conflation of the dominant models of hypothesis testing. 8,9 The setting of theoretical type 1 (α) and type 2 (β) error rates in the Neyman and Pearson model envisions the frequency of error 'in the long run of experience' (experimental repetition) given randomness and independence of sample means from two juxtaposed probability distributions. A priori two P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone 3…”

Section: What Does a P-value Not Measure? Error Ratesmentioning

confidence: 99%

See 2 more Smart Citations

P‐value. What value?

O’Donnell

2018

Internal Medicine Journal

View full text Add to dashboard Cite

Section: What Is a P‐value And What Does It Measure?mentioning

confidence: 99%

Section: What Is a P-value And What Does It Measure?mentioning

confidence: 99%

Section: What Does a P-value Not Measure? Error Ratesmentioning

confidence: 99%

See 1 more Smart Citation

P‐value. What value?

O’Donnell

2018

Internal Medicine Journal

View full text Add to dashboard Cite

“…Unlike in the model-based approach, no model distributional assumptions were needed to convert y into a random variable. Hence, randomness of the sampling design is a mandatory requirement under Neyman's framework because it is the sole basis for the probabilistic treatment of the results during data analysis (Fienberg & Tanur, 1996). …”

Section: Design-based Inferential Frameworkmentioning

confidence: 99%

Alternative Model-Based and Design-Based Frameworks for Inference From Samples to Populations: From Polarization to Integration

Sterba

2009

Multivariate Behavioral Research

View full text Add to dashboard Cite

A model-based framework, due originally to R. A. Fisher, and a design-based framework, due originally to J. Neyman, offer alternative mechanisms for inference from samples to populations. We show how these frameworks can utilize different types of samples (nonrandom or random vs. only random) and allow different kinds of inference (descriptive vs. analytic) to different kinds of populations (finite vs. infinite). We describe the extent of each framework's implementation in observational psychology research. After clarifying some important limitations of each framework, we describe how these limitations are overcome by a newer hybrid model/designbased inferential framework. This hybrid framework allows both kinds of inference to both kinds of populations, given a random sample. We illustrate implementation of the hybrid framework using the High School and Beyond data set.Nonrandom sampling involves selecting units (e.g., persons) with unknown probabilities of selection from a finite population of units. This finite population may be poorly defined (e.g., all persons who saw a flier posted on a community bulletin board) or well defined (e.g., all children attending licensed daycare centers in Dayton, Ohio). Nonrandom, or purposive, sampling is common in observational psychology research. For example, 76% of observational studies in 2006 issues of Journal of Personality and Social Psychology, Developmental Psychology, Journal of Abnormal Psychology, and Journal of Educational Psychology used nonrandom samples (Sterba, Prinstein, & Nock, 2008). Psychologists often raise the following two questions about nonrandom samples in observational research (e.g., Jaffe, 2005;Peterson, 2001;Sears, 1986;Serlin, Wampold, & Levin, 2003;Sherman, Buddie, Dragan, End, & Finney, 1999; Siemer & Joorman, 2003;Wintre, North, & Sugar, 2001). Can statistical inferences be made from nonrandom samples; if so, under what conditions and to what population?2. Do inferences made from nonrandom samples differ from those possible under random sampling?According to some psychology research methods texts, the answer to the first question is no: "Although these purposive sampling methods are more practical than formal probability sampling, they are not backed by a statistical logic that justifies formal generalizations" (Shadish, Cook, & Campbell, 2002, pp. 24, 356; see also Cook & Campbell, 1979, pp. 72-73 1982, pp. 255, 158-166).Again consulting psychology research methods texts, the answer to the second question remains unclear. For example, Shadish et al. (2002) note that randomly selecting units-that is, sampling units with known probabilities of selection from a well-defined finite population of units-facilitates generalization from those sample units to the finite population by ensuring a "match between sample and population distributions on measured and unmeasured attributes within known limits of sampling error" (p. 343; see also Cook & Campbell, 1979, p. 75). 2 But specifics are not provided as to whether the known probabilities ...

show abstract

“…Experienced statisticians are often unable to help, or even give conflicting advice, because they are unfamiliar with typical zoo study designs and have been trained in classic agronomic-derived statistical techniques based on very large sample sizes, controlled conditions and multiple replicates [Fienberg and Tanur, 1966;Kuhar, 2006]. In addition, among published accounts of zoo-based research the variety of statistical tests applied to similar data sets derived in similar ways is almost as numerous as the articles themselves and many articles, sadly, include serious statistical errors [Kuhar, 2006].…”

Section: Introductionmentioning

confidence: 99%

BIAZA statistics guidelines: toward a common application of statistical tests for zoo research

Plowman

2008

Zoo Biology

View full text Add to dashboard Cite

Zoo research presents many statistical challenges, mostly arising from the need to work with small sample sizes. Efforts to overcome these often lead to the misuse of statistics including pseudoreplication, inappropriate pooling, assumption violation or excessive Type II errors because of using tests with low power to avoid assumption violation. To tackle these issues and make some general statistical recommendations for zoo researchers, the Research Group of the British and Irish Association of Zoos and Aquariums (BIAZA) conducted a workshop. Participants included zoo-based researchers, university academics with zoo interests and three statistical experts. The result was a BIAZA publication Zoo Research Guidelines: Statistics for Typical Zoo Datasets (Plowman [2006] Zoo research guidelines: statistics for zoo datasets. London: BIAZA), which provides advice for zoo researchers on study design and analysis to ensure appropriate and rigorous use of statistics. The main recommendations are: (1) that many typical zoo investigations should be conducted as single case/small N randomized designs, analyzed with randomization tests, (2) that when comparing complete time budgets across conditions in behavioral studies, G tests and their derivatives are the most appropriate statistical tests and (3) that in studies involving multiple dependent and independent variables there are usually no satisfactory alternatives to traditional parametric tests and, despite some assumption violations, it is better to use these tests with careful interpretation, than to lose information through not testing at all. The BIAZA guidelines were recommended by American Association of Zoos and Aquariums (AZA) researchers at the AZA Annual Conference in Tampa, FL, September 2006, and are free to download from www.biaza.org.uk.

show abstract

Reconsidering the Fundamental Contributions of Fisher and Neyman on Experimentation and Sampling

Cited by 41 publications

References 34 publications

P‐value. What value?

P‐value. What value?

Alternative Model-Based and Design-Based Frameworks for Inference From Samples to Populations: From Polarization to Integration

BIAZA statistics guidelines: toward a common application of statistical tests for zoo research

Contact Info

Product

Resources

About