Inference for Non-random Samples

Abstract

Observational data are often analysed as if they had resulted from a controlled study, and yet the tacit assumption of randomness can be crucial for the validity of inference. We take some simple statistical models and supplement them by adding a parameter which re¯ects the degree of non-randomness in the sample. For a randomized study is known to be 0. We examine the pro®le log-likelihood for and the sensitivity of inference to small nonzero values of . Particular models cover the analysis of survey data with item nonresponse, the paired comparison t-test and two group comparisons using observational data with covariates. Some practical examples are discussed. Allowing for sampling bias increases the uncertainty of estimation and weakens the signi®cance of treatment effects, sometimes substantially so.

Confounding and Collapsibility in Causal Inference

“…It is an unfortunate but true fact that many important causal questions are simply not answerable, at least not without employing assumptions that are untestable given current technology, Examples of such assumptions include assumptions of no confounding, as discussed in the following sections, assumptions about independence of unit-specific susceptibilities or responses, and various distributional assumptions (Copas, 1973;Rubin, 1978Rubin, , 1991Holland, 1986;Heckman and Hotz, 1989;Robins and Greenland, 1989;Sobel, 1995;Rosenbaum, 1995;Copas and Li, 1997). Inferences from counterfactual approaches properly reflect this harsh epistemic reality when they display sensitivity to such assumptions.…”

“…It almost always remains logically possible that this set is insufficient because some confounder essential for sufficiency has not been recorded; thus, causal inferences from observational studies almost always hinge on subject-matter priors ("judgements") about what may be missing from the set. Sensitivity of results to possible unmeasured confounders can be assessed via formal sensitivity analysis (Rosenbaum, 1995;Copas and Li, 1997;Robins, Rotnitzky, and Scharfstein, 1999).…”

Randomness and Inference in Medical and Public Health Research

“…Statistical inference is practical and useful only when the goal is to make inferences about the population based on the sample and is warranted only for studies with a random sample. Without one, the necessary assumptions are not met and inference may not be appropriate (Smith, 1983, andCopas andLi, 1997, Although p-values and confidence intervals are commonly reported in the medical and public health literature, many scientists lack sufficient understanding to interpret them correctly (Wulff et al, 1987). Both are inferential statistics and are meaningful only with respect to making statements about a larger population based on a random sample taken from it.…”

Uncertainty intervals for regression parameters with non-ignorable missingness in the outcome

“…The uncertainty intervals that we introduce may be used as a tool for sensitivity analysis as we illustrate in our case study. Our approach is in this respect closely related to the one proposed by Copas and Li (1997), as the selection parameter θ in their paper is a transformation of ρ. Copas and Li (1997) build a profile log likelihood for θ in order to carry out a sensitivity analysis.…”

“…Our approach is in this respect closely related to the one proposed by Copas and Li (1997), as the selection parameter θ in their paper is a transformation of ρ. Copas and Li (1997) build a profile log likelihood for θ in order to carry out a sensitivity analysis. Similar models and methods are used for sensitivity analysis to publication bias in meta analysis (Copas 2013, Henmi et al 2007.…”