This paper presents an integrated framework for estimation and inference from generalized linear models using adjusted score equations that result in mean and median bias reduction. The framework unifies theoretical and methodological aspects of past research on mean bias reduction and accommodates, in a natural way, new advances on median bias reduction. General expressions for the adjusted score functions are derived in terms of quantities that are readily available in standard software for fitting generalized linear models. The resulting estimating equations are solved using a unifying quasi-Fisher scoring algorithm that is shown to be equivalent to iteratively re-weighted least squares with appropriately adjusted working variates. Formal links between the iterations for mean and median bias reduction are established. Core model invariance properties are used to develop a novel mixed adjustment strategy when the estimation of a dispersion parameter is necessary. It is also shown how median bias reduction in multinomial logistic regression can be done using the equivalent Poisson log-linear model. The estimates coming out from mean and median bias reduction are found to overcome practical issues related to infinite estimates that can occur with positive probability in generalized linear models with multinomial or discrete responses, and can result in valid inferences even in the presence of a high-dimensional nuisance parameter.

No abstract

No abstract

For regular parametric problems, we show how median centering of the maximum likelihood estimate can be achieved by a simple modification of the score equation. For a scalar parameter of interest, the estimator is equivariant under interest respecting parameterizations and third-order median unbiased. With a vector parameter of interest, componentwise equivariance and thirdorder median centering are obtained. Like Firth's (1993, Biometrika) implicit method for bias reduction, the new method does not require finiteness of the maximum likelihood estimate and is effective in preventing infinite estimates. Simulation results for continuous and discrete models, including binary and beta regression, confirm that the method succeeds in achieving componentwise median centering and in solving the infinite estimate problem, while keeping comparable dispersion and the same approximate distribution as its main competitors.

We consider inference on a vector-valued parameter of interest in a linear exponential family, in the presence of a finite-dimensional nuisance parameter. Based on higher order asymptotic theory for likelihood, we propose a directional test whose p-value is computed using one-dimensional integration. For discrete responses this extends the development of Davison et al. (2006), and some of our examples concern testing in contingency tables. For continuous responses the work extends the directional test of Cheah et al. (1994). Examples and simulations illustrate the high accuracy of the method, which we compare with the usual likelihood ratio test and with an adjusted version due to Skovgaard (2001). In high-dimensional settings, such as covariance selection, the approach works essentially perfectly, whereas its competitors can fail catastrophically.