2013
DOI: 10.1287/isre.2013.0480
|View full text |Cite
|
Sign up to set email alerts
|

Research Commentary—Too Big to Fail: Large Samples and the p-Value Problem

Abstract: T he Internet has provided IS researchers with the opportunity to conduct studies with extremely large samples, frequently well over 10,000 observations. There are many advantages to large samples, but researchers using statistical inference must be aware of the p-value problem associated with them. In very large samples, p-values go quickly to zero, and solely relying on p-values can lead the researcher to claim support for results of no practical significance. In a survey of large sample IS research, we foun… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

15
541
1
5

Year Published

2015
2015
2022
2022

Publication Types

Select...
9
1

Relationship

0
10

Authors

Journals

citations
Cited by 810 publications
(562 citation statements)
references
References 49 publications
15
541
1
5
Order By: Relevance
“…Therefore, based on recent work on this topic Lin et al 2013, in Table 5, we present the effect size and 95% confidence interval for both direct effects and moderation effects. Specifically, one unit increase in RM, WIP, and FG inventory, ROA increases by 0.86, 0.48, and 0.43 percent, respectively.…”
Section: Analysis and Resultsmentioning
confidence: 99%
“…Therefore, based on recent work on this topic Lin et al 2013, in Table 5, we present the effect size and 95% confidence interval for both direct effects and moderation effects. Specifically, one unit increase in RM, WIP, and FG inventory, ROA increases by 0.86, 0.48, and 0.43 percent, respectively.…”
Section: Analysis and Resultsmentioning
confidence: 99%
“…The low p-values are counter-intuitive given the low D-statistics. We observe from the last columns of Table 5 and Table 6 that these distributions have large sample size, suggesting that the low p-values are an artifact of a large number of samples [15]. To support this claim, we compare the EDFs with their respective input CDFs.…”
Section: Validationmentioning
confidence: 88%
“…Due to the large sample size, p values were not used for between-model comparisons, as even negligible differences are likely to be statistically significant. 34 Instead, as adjunctive measures of model performance, the Akaike information criterion (AIC), Bayesian information criterion (BIC), pseudo R 2 , and shrinkage coefficients were also calculated. Smaller AIC and BIC values and larger pseudo R 2 and shrinkage coefficients indicate better model performance.…”
Section: Discussionmentioning
confidence: 99%