Bias in product availability estimates from contraceptive outlet surveys: Evidence from the Consumer’s Market for Family Planning (CM4FP) study

Abstract

Area-based sampling approaches designed to capture pharmacies, drug shops, and other non-facility service delivery outlets are critical for accurately measuring the contraceptive service environment in contexts of increasing de-medicalization of contraceptive commodities and services. Evidence from other disciplines has demonstrated area-based estimates may be biased if there is spatial heterogeneity in product distribution, but this bias has not yet been assessed in the context of contraceptive supply estimates. The Consumer’s Marker for Family Planning (CM4FP) study conducted censuses and product audits of contraceptive outlets across 12 study sites and 2–3 rounds of quarterly data collection in Kenya, Nigeria, and Uganda. We assessed bias in estimates of contraceptive product availability by comparing estimates from simulations of area-based sampling approaches with census counts among all audited facilities for each study site and round of data collection. We found evidence of bias in estimates of contraceptive availability generated from simulated area-based sampling. Within specific study sites and rounds, we observed biased sampling estimates for several but not all contraceptive method types, with bias more likely to occur in sites with heterogeneity in both spatial distribution of outlets and product availability within outlets. In simulations varying size of enumeration areas (EA) and number of outlets sampled per EA, we demonstrated that the likelihood of substantial bias decreases as EA size decreases and as the number of outlets sampled per EA increases. Straightforward approaches such as increasing sample size per EA or applying statistical weights may be used to reduce area-based sampling bias, indicating a pragmatic way forward to improve estimates where design-based sampling is infeasible. Such approaches should be considered in development of improved methods for area-based estimates of contraceptive supply-side environments.