A Unified Evaluation of Two-Candidate Ballot-Polling Election Auditing Methods

Abstract

Counting votes is complex and error-prone. Several statistical methods have been developed to assess election accuracy by manually inspecting randomly selected physical ballots. Two 'principled' methods are risk-limiting audits (RLAs) and Bayesian audits (BAs). RLAs use frequentist statistical inference while BAs are based on Bayesian inference. Until recently, the two have been thought of as fundamentally different. We present results that unify and shed light upon 'ballot-polling' RLAs and BAs (which only require the ability to sample uniformly at random from all cast ballot cards) for two-candidate plurality contests, the are building blocks for auditing more complex social choice functions, including some preferential voting systems. We highlight the connections between the methods and explore their performance. First, building on a previous demonstration of the mathematical equivalence of classical and Bayesian approaches, we show that BAs, suitably calibrated, are risk-limiting. Second, we compare the efficiency of the methods across a wide range of contest sizes and margins, focusing on the distribution of sample sizes required to attain a given risk limit. Third, we outline several ways to improve performance and show how the mathematical equivalence explains the improvements.

ALPHA: Audit that Learns from Previously Hand-Audited Ballots

“…Columns 10:16 give the mean sample sizes to reject the null when the sample is allowed to expand to comprise the whole population of 20,000 ballot cards. The five bottom rows are from supplementary materials in [8] available at https://github.com/dvukcevic/AuditAnalysis/blob/master/combined_tab les/n%3D020000_m%3D02000_p%3D0.500_replacement%3DFalse_step%3D1/unconditional_mean_with_rec ount_addin.csv#L27 (last visited 1 February 2022). The "Bayesian" test uses a risk-maximizing prior [24] (in this case, a point mass at 1/2 mixed with a uniform on (1/2, 1]), which makes it risk-limiting.…”

“…The methods listed include the bestperforming method in RiLACS [27] that uses an explicit alternative η (a priori Kelly), the best-performing method in RiLACS that does not use a pre-specified alternative (SqKelly), Wald's SPRT for sampling without replacement (the analogue of BRAVO for sampling without replacement), and ALPHA with the truncated shrinkage estimator for a variety of values of d. Methods that use an explicit alternative (a priori Kelly, SPRT, ALPHA) were tested using a range of values of η. Kaplan's martingale [19] was not included because it is expensive to compute, numerically unstable, and performs comparably to some of the methods studied by [27], such as dKelly. The election parameters N , θ, and η were chosen to make the simulations commensurable with [8].…”

“…Thus, for a population of N = 20,000 ballot cards, an election official might elect to conduct a full hand count if the audit sample becomes larger than 2,000 cards, 10% of the population. The entries for Bayesian, BRAVO, and ClipAudit are derived from Table 2 of [8] by adding 18, 000 × (1 − power) to the entries, since that table was calculated by capping the sample size at 2, 000. The columns in Table 2 for n ≤ N = 20,000 do not restrict the sample size; they show the expected sample size of audits when the audit is allowed to escalate one card at a time, potentially to a full hand count.…”

“…The "Bayesian" test uses a risk-maximizing prior [24] (in this case, a point mass at 1/2 mixed with a uniform on (1/2, 1]), which makes it risk-limiting. ClipAudit [13] is calibrated in a way that almost limits the risk; in simulations it was 5.1% [8]. The smallest average sample size in each column, omitting ClipAudit, is in bold font.…”

RiLACS: Risk-Limiting Audits via Confidence Sequences

“…The earliest work on RLAs did not use anytime p-values [6,7], but since about 2009, most RLA methods have used anytime p-values to conduct sequential hypothesis tests [8,9,3,1,10]. An anytime p-value is a sequence of p-values (p t ) N t=1 with the property that under some null hypothesis…”