Confidence interval construction for proportion difference in small‐sample paired studies

Abstract

Paired dichotomous data may arise in clinical trials such as pre-/post-test comparison studies and equivalence trials. Reporting parameter estimates (e.g. odds ratio, rate difference and rate ratio) along with their associated confidence interval estimates becomes a necessity in many medical journals. Various asymptotic confidence interval estimators have long been developed for differences in correlated binary proportions. Nevertheless, the performance of these asymptotic methods may have poor coverage properties in small samples. In this article, we investigate several alternative confidence interval estimators for the difference between binomial proportions based on small-sample paired data. Specifically, we consider exact and approximate unconditional confidence intervals for rate difference via inverting a score test. The exact unconditional confidence interval guarantees the coverage probability, and it is recommended if strict control of coverage probability is required. However, the exact method tends to be overly conservative and computationally demanding. Our empirical results show that the approximate unconditional score confidence interval estimators based on inverting the score test demonstrate reasonably good coverage properties even in small-sample designs, and yet they are relatively easy to implement computationally. We illustrate the methods using real examples from a pain management study and a cancer study.

Statistical distribution of the difference of two proportions

“…With respect to published work in the Statistics in Medicine, see, e.g. References [1][2][3][4][5][6]. In all of these papers, the statistical distribution of the differences of proportions appears to have been approximated by some simple form, e.g.…”

On Confidence Interval Construction for Establishing Equivalence of Two Binary-Outcome Treatments in Matched-Pair Studies in the Presence of Incomplete Data

“…Empirical results given by Tango [18] and Newcombe [12] demonstrated that the profile-likelihood-based and score-test-based confidence intervals performed satisfactorily in large sample designs. Tang et al [16] empirically demonstrated that Tango score confidence interval performed unsatisfactorily in small-sample designs and developed an exact unconditional and an approximate unconditional confidence intervals for proportion difference in small-sample paired studies. However, all the aforementioned work were confined to matched-pair data without incomplete data.…”

“…Define X = 1 if a patient vomited at least once during the last six-hour period (i.e., hour 18-24) after receiving MPRED; = 0 otherwise, and Y = 1 if a patient vomited at least once during the last six-hour period after receiving METCLO; = 0 otherwise. It was reported that amongst the 157 eligible patients, (i) 115 received both treatments in the two cycles (i.e., with both X and Y being observed); (ii) 16 received only MPRED for the first cycle but not METCLO for the second cycle (i.e., with only X being observed); and (iii) 26 received only METCLO for the first cycle but not MPRED for the second cycle (i.e., with only Y being observed). For scenario (i), it was reported that 106 patients experienced at least one vomiting in either treatment (i.e., 115 − 106 = 9 patients had no vomiting experience).…”

A comparison of methods for the construction of confidence interval for relative risk in stratified matched‐pair designs

“…Tango [7] presented the profile-likelihood-based confidence interval and the score-test-based confidence interval for risk difference in paired-design studies on the basis of asymptotic theories, and some empirical results given in Tango [8] and Newcombe [9] indicated that the profile-likelihood-based confidence interval and the score-test-based confidence interval perform satisfactorily in large-sample designs. Tang et al [10] noted that Tango's [7] confidence intervals perform unsatisfactorily in small-sample designs. Therefore, Tang et al [10] developed an exact unconditional confidence interval and an approximate unconditional confidence interval for proportion difference in small-sample paired studies.…”

“…Tang et al [10] noted that Tango's [7] confidence intervals perform unsatisfactorily in small-sample designs. Therefore, Tang et al [10] developed an exact unconditional confidence interval and an approximate unconditional confidence interval for proportion difference in small-sample paired studies. However, all the above cited works are confined to a single 2×2 table, and there is little work for confidence interval construction in multiple 2×2 tables.…”