Optimal Data Acquisition for Statistical Estimation

Abstract

We consider a data analyst's problem of purchasing data from strategic agents to compute an unbiased estimate of a statistic of interest. Agents incur private costs to reveal their data and the costs can be arbitrarily correlated with their data. Once revealed, data are verifiable. This paper focuses on linear unbiased estimators. We design an individually rational and incentive compatible mechanism that optimizes the worst-case mean-squared error of the estimation, where the worst-case is over the unknown correlation between costs and data, subject to a budget constraint in expectation. We characterize the form of the optimal mechanism in closed-form. We further extend our results to acquiring data for estimating a parameter in regression analysis, where private costs can correlate with the values of the dependent variable but not with the values of the independent variables. Full Paper: https://arxiv.org/abs/1711.01295

Prior-free Data Acquisition for Accurate Statistical Estimation

“…. , (c n , z n ) and the internal randomness of the mechanisms M. Roth and Schoenebeck [2012] and Chen et al [2018a] have also considered the design of joint survey and estimation mechanism for statistical estimation. The main differences between their model and our model are: (1) they assume the marginal cost distribution is known to the data analyst, while our data analyst doesn't have such information, (2) they have the same survey mechanism for all individuals, while we consider an online setting where the analyst can adaptively change the survey mechanism, and (3) they only consider the estimation of mean, while we also investigate the estimation of confidence intervals.…”

“…In this section, we first show that we can easily extend known results on one-shot truthful mechanisms to achieve truthfulness and individual rationality for a sequence of survey mechanisms M. Then, we introduce the formulation proposed by Chen et al [2018a] for obtaining the optimal unbiased estimator of population mean when the cost distribution is known to the analyst. Later in Section 5 we will use this known cost case as our benchmark for evaluating the performance of our optimal unbiased estimator when the cost distribution is unknown.…”

“…Later in Section 5 we will use this known cost case as our benchmark for evaluating the performance of our optimal unbiased estimator when the cost distribution is unknown. While this optimal unbiased estimator has been studied by Chen et al [2018a], their optimal mechanism requires the cost distribution to be regular. This is often not satisfied when we consider the costs of a finite set of data holders.…”

“…The problem of purchasing data for unbiased estimation of population mean was first formulated by Roth and Schoenebeck [2012] and then further studied by Chen et al [2018a]. Both works however assume that the cost distribution is known to the analyst and aim at obtaining an optimal unbiased estimator with minimum worst-case variance for population mean, where the worst-case is over all data-cost distributions consistent with the known cost distribution, subject to an expected budget constraint.…”

The Markovian Price of Information

“…There is also a large body of work in related models where information has a price [28,10,32,25,14,1,13,12]. Finally, as discussed in the introduction, the works in [33] and [37] are directly relevant to this paper.…”