Location-adjusted Wald statistics for scalar parameters

Abstract

Inference about a scalar parameter of interest is a core statistical task that has attracted immense research in statistics. The Wald statistic is a prime candidate for the task, on the grounds of the asymptotic validity of the standard normal approximation to its finite-sample distribution, simplicity and low computational cost. It is well known, though, that this normal approximation can be inadequate, especially when the sample size is small or moderate relative to the number of parameters. A novel, algebraic adjustment to the Wald statistic is proposed, delivering significant improvements in inferential performance with only small implementation and computational overhead, predominantly due to additional matrix multiplications. The Wald statistic is viewed as an estimate of a transformation of the model parameters and is appropriately adjusted, using either maximum likelihood or reduced-bias estimators, bringing its expectation asymptotically closer to zero. The location adjustment depends on the expected information, an approximation to the bias of the estimator, and the derivatives of the transformation, which are all either readily available or easily obtainable in standard software for a wealth of models. An algorithm for the implementation of the location-adjusted Wald statistics in general models is provided, as well as a bootstrap scheme for the further scale correction of the location-adjusted statistic. Ample analytical and numerical evidence is presented for the adoption of the location-adjusted statistic in prominent modelling settings, including inference about log-odds and binomial proportions, logistic regression in the presence of nuisance parameters, beta regression, and gamma regression. The location-adjusted Wald statistics are used for the construction of significance maps for the analysis of multiple sclerosis lesions from MRI data.

Bias‐reduced estimators of conditional odds ratios in matched case‐control studies with unmatched confounding

“…It turns out that the first‐order bias of $${\hat{\psi}}_{j}$$ (see Kosmidis & Firth, 2010, Section 4.3., Remark 3) can be written in terms of the first‐order bias of $$\hat{\theta }_j$$, the first two derivatives of the function exp and the inverse of the expected information matrix. The first‐order bias of general transformations of MLE is given in Di Caterina and Kosmidis (2019) and is used to derive a simple location adjustment for common Wald statistics that considerably improves the performance of Wald‐type inference. In particular when using the function exp , by Di Caterina and Kosmidis (2019, Expression 6) or Kosmidis and Firth (2010, Section 4.3., Remark 3), $$$$\begin{equation*} E(\hat{\psi }_j-\psi _j )=B_{\hat{\psi }_j}(\bm \psi )+O(n^{-2})\mbox{, }j=1,\ldots ,p, \end{equation*}$$$$where …”

“…It turns out that the first-order bias of ψ𝑗 (see Kosmidis & Firth, 2010, Section 4.3., Remark 3) can be written in terms of the first-order bias of θ𝑗 , the first two derivatives of the function exp and the inverse of the expected information matrix. The first-order bias of general transformations of MLE is given in Di Caterina and Kosmidis (2019) and is used to derive a simple location adjustment for common Wald statistics that considerably improves the performance of Wald-type inference. In particular when using the function exp, by Di Caterina and Kosmidis (2019, Expression 6) or Kosmidis and Firth (2010, Section 4.3., Remark 3),…”

“…The first‐order bias of general transformations of MLE is given in Di Caterina and Kosmidis (2019) and is used to derive a simple location adjustment for common Wald statistics that considerably improves the performance of Wald‐type inference. In particular when using the function exp , by Di Caterina and Kosmidis (2019, Expression 6) or Kosmidis and Firth (2010, Section 4.3., Remark 3), $$$$\begin{equation*} E(\hat{\psi }_j-\psi _j )=B_{\hat{\psi }_j}(\bm \psi )+O(n^{-2})\mbox{, }j=1,\ldots ,p, \end{equation*}$$$$where …”

Mean and median bias reduction: A concise review and application to adjacent-categories logit models

“…Di Caterina and Kosmidis (2019) show that there is a simple way to derive the mean bias of h(θ * ) for any three-times differentiable function h : C → D, with C ⊂ p and D ⊂ , where θ * is a mBR estimator of θ with O(N −2 ) bias. In particular, Di Caterina and Kosmidis (2019) show that the estimator h(θ * ) of ζ = h(θ) has mean bias…”