A Note on Controlling Significance Levels in Stepwise Regression

Kupper, Lawrence L.; Stewart, J R.; Williams, Kenneth

doi:10.1093/oxfordjournals.aje.a112200

Cited by 29 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A stepwise logistic regression analysis was performed to determine which of these 7 measures made independent contributions in discriminating between patients treated with placebo and patients treated with an active drug. Because we anticipated that several of the outcome measures would make such a contribution and because we wanted to reduce the risk of Type I1 error in this analysis, the significance level for variables to enter and to remain in the logistic regression was set at 0.15 (5). The extent of discrimination achieved was measured by a concordance index (c), representing the probability that a randomly selected pair of observations (one from the placebo group and the other from the treatment group) would be correctly assigned using the logistic discriminant function.…”

Section: Methodsmentioning

confidence: 99%

Which traditional measures should be used in rheumatoid arthritis clinical trials?

Anderson

Felson

Meenan

et al. 1989

Arthritis & Rheumatism

133

View full text Add to dashboard Cite

It is standard practice to use multiple outcome measures in rheumatoid arthritis (RA) clinical trials. Because of this, multiple testing is usually done, and there is confusion in the interpretation of the results. It is not clear which measures are the most sensitive to detecting improvement and which provide independent information. To address these questions, we conducted an analysis of pooled raw data from 3 placebo-controlled RA trials: one of methotrexate, another of oral and injectable gold, and the third of low-dose and high-dose D-penicillamine. The results show that the joint tenderness count, erythrocyte sedimentation rate, grip strength, and physician assessment of disease activity perform best in RA clinical trials and cover the domain of change measured by traditional outcome measures. Other clinical measures may provide little additional useful information for a standard therapeutic trial of up to 6 months duration, and some measures, such as the proximal interphalangeal joint circumference, hemoglobin level, and 50-foot walking time, can be eliminated from such trials.

show abstract

Section: Methodsmentioning

confidence: 99%

Which traditional measures should be used in rheumatoid arthritis clinical trials?

Anderson

Felson

Meenan

et al. 1989

Arthritis & Rheumatism

133

View full text Add to dashboard Cite

show abstract

“…This procedure is a compromise between biased estimation, which may result from an overly parsimonious specification (such as Lave and Seskin's, for example), and spurious significance, which can result with stepwise techniques when a large number of variables is available to the specification. 21 The use of the minimum residual error is one criterion for such a compromise, 22 and has been adopted for the analysis that follows. A number of other pollutants (trace metals) and variables (including climatic variables) were tested in exploratory regressions and discarded since they were not even marginally statistically significant and showed no evidence of interactions with the main variables of interest.…”

mentioning

confidence: 99%

Sulfur Oxides, Particulates, and Human Mortality: Synopsis of Statistical Correlations

Lipfert¹

1980

Journal of the Air Pollution Control Association

View full text Add to dashboard Cite

“…The 5% alpha-level was the preference for significance testing in the bivariate analyses, while a 5% alpha-entry-level and a 10% alpha-removal-level were the norm for studies that used stepwise routines. These significance levels, however, have been found to be too high for minimizing type I error [2], and also too low for minimizing type II error and attaining good predictive models [1,3,21,22]. More disturbingly, 60% of the studies that used stepwise procedures failed to specify the algorithm and/or the significance levels, and 15% (33/219) of the studies that utilized multivariable models did not mention how the variables were selected.…”

Section: Discussionmentioning

confidence: 99%

“…In particular, automated techniques have been especially popular in the past, perhaps because they are discussed in nearly all elementary textbooks on applied statistics and implemented in many commercial statistical software packages. Such techniques are however notorious for underestimated standard errors and inflated significance levels, inclusion of irrelevant variables and exclusion of authentic predictors, and unstable solutions even with minor changes in the data [2,4,8,13-17]. …”

Section: Introductionmentioning

confidence: 99%

A survey of variable selection methods in two Chinese epidemiology journals

Liao

Lynn

2010

BMC Med Res Methodol

View full text Add to dashboard Cite

BackgroundAlthough much has been written on developing better procedures for variable selection, there is little research on how it is practiced in actual studies. This review surveys the variable selection methods reported in two high-ranking Chinese epidemiology journals.MethodsArticles published in 2004, 2006, and 2008 in the Chinese Journal of Epidemiology and the Chinese Journal of Preventive Medicine were reviewed. Five categories of methods were identified whereby variables were selected using: A - bivariate analyses; B - multivariable analysis; e.g. stepwise or individual significance testing of model coefficients; C - first bivariate analyses, followed by multivariable analysis; D - bivariate analyses or multivariable analysis; and E - other criteria like prior knowledge or personal judgment.ResultsAmong the 287 articles that reported using variable selection methods, 6%, 26%, 30%, 21%, and 17% were in categories A through E, respectively. One hundred sixty-three studies selected variables using bivariate analyses, 80% (130/163) via multiple significance testing at the 5% alpha-level. Of the 219 multivariable analyses, 97 (44%) used stepwise procedures, 89 (41%) tested individual regression coefficients, but 33 (15%) did not mention how variables were selected. Sixty percent (58/97) of the stepwise routines also did not specify the algorithm and/or significance levels.ConclusionsThe variable selection methods reported in the two journals were limited in variety, and details were often missing. Many studies still relied on problematic techniques like stepwise procedures and/or multiple testing of bivariate associations at the 0.05 alpha-level. These deficiencies should be rectified to safeguard the scientific validity of articles published in Chinese epidemiology journals.

show abstract

A Note on Controlling Significance Levels in Stepwise Regression

Cited by 29 publications

References 0 publications

Which traditional measures should be used in rheumatoid arthritis clinical trials?

Which traditional measures should be used in rheumatoid arthritis clinical trials?

Sulfur Oxides, Particulates, and Human Mortality: Synopsis of Statistical Correlations

A survey of variable selection methods in two Chinese epidemiology journals

Contact Info

Product

Resources

About