A framework for optimizing COVID-19 testing policy using a Multi Armed Bandit approach

Abstract

Testing is an important part of tackling the COVID-19 pandemic. Availability of testing is a bottleneck due to constrained resources and effective prioritization of individuals is necessary. Here, we discuss the impact of different prioritization policies on COVID-19 patient discovery and the ability of governments and health organizations to use the results for effective decision making. We suggest a framework for testing that balances maximal discovery of positive individuals with the need for population-based surveillance aimed at understanding disease spread and characteristics. This framework draws from similar approaches to prioritization in the domain of cyber-security based on ranking individuals using a risk score and then reserving a portion of the capacity for random sampling. This approach is an application of Multi-Armed-Bandits maximizing exploration/exploitation of the underlying distribution. We find that individuals can be ranked for effective testing using a few simple features, and that ranking them using such models we can capture 65% (CI: 64.7%-68.3%) of the positive individuals using less than 20% of the testing capacity or 92.1% (CI: 91.1%-93.2%) of positives individuals using 70% of the capacity, allowing reserving a significant portion of the tests for population studies. Our approach allows experts and decision makers to tailor the resulting policies as needed allowing transparency into the ranking policy and the ability to understand the disease spread in the population and react quickly and in an informed manner.

Containing a spread through sequential learning: to exploit or to explore?

“…(iv) Logistic Regression: We use ideas presented in [51], where simple classifiers were proposed based on the features of real data. In our setting, we choose the classifier to be based on logistic regression, and we define the feature of node i as X i (t) = [1, n i (t) + ] T .…”

“…Nodes are tested based on some scores obtained by the sequential learning framework, but no fundamental probabilities of the states of nodes were revealed. Different from [51], [52], our approach is model-based and we observe novel exploration-exploitation tradeoffs that arise not due to a lack of knowledge about the model or network, but rather because the set of infected nodes is unknown and evolves with time. We can also utilize knowledge about both the model and the contact network to devise a probabilistic framework for decision making.…”

“…We now distinguish our work from testing strategies that combine exploitation and exploration in some form [50], [51], [52]. Through a theoretical approach, [50] models the testing problem as a partially observable Markov decision process (POMDP).…”

“…This is a very special case that is hard to justify in practice and it is not clear how one could go beyond this assumption. On the other hand, [51] proposes a heuristic by implementing classical learning methods such as Linear support vector machine (SVM) and Polynomial SVM to rank nodes based on a notion of risk score (constructed by real-data) while reserving a portion of the test budget for random testing which can be understood as exploration. No spread model or contact network is assumed.…”

Whom to Test? Active Sampling Strategies for Managing COVID-19

“…This model serves as a basis for the ABM that we developed in this work. Also in contemporaneous work, [28] present a framework that uses historical data to build a classifier that computes a "risk score", which is the basis of determining how to combine exploration and exploitation. While the idea of using bandit-based algorithms for testing is important, the challenge is in developing specific methods that address all the complexities noted previously.…”