Multi-Objective Model-based Reinforcement Learning for Infectious Disease Control

Wan, Runzhe; Zhang, Xinyu; Song, Rui

doi:10.1145/3447548.3467303

Cited by 18 publications

(9 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Choosing a larger c will result in a greater AES, thus a shorter duration, as the longer experiments incurs higher costs. The impact of c has been studied in Figure 4 in Wan et al (2023).…”

Section: Utility Theory: Bayesian Optimal Effect Sizementioning

confidence: 99%

“…It can also compromise the statistical power of the experiment, leading to inaccuracies in detecting meaningful effects (Ramsey and Schafer 2012;Lenth 2001). Conversely, if the sample size is selected as too large, it could unnecessarily extend the duration of the experiment and inflate the associated costs, such as the human and hardware resources and opportunity costs (Wan et al 2023). Therefore, selecting an appropriate sample size is imperative to maintain a balance between decision accuracy and resource utilization.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effect Size Estimation for Duration Recommendation in Online Experiments: Leveraging Hierarchical Models and Objective Utility Approaches

Liu,

Wan,

McQueen

et al. 2024

AAAI

View full text Add to dashboard Cite

The selection of the assumed effect size (AES) critically determines the duration of an experiment, and hence its accuracy and efficiency. Traditionally, experimenters determine AES based on domain knowledge. However, this method becomes impractical for online experimentation services managing numerous experiments, and a more automated approach is hence of great demand. We initiate the study of data-driven AES selection in for online experimentation services by introducing two solutions. The first employs a three-layer Gaussian Mixture Model considering the heteroskedasticity across experiments, and it seeks to estimate the true expected effect size among positive experiments. The second method, grounded in utility theory, aims to determine the optimal effect size by striking a balance between the experiment's cost and the precision of decision-making. Through comparisons with baseline methods using both simulated and real data, we showcase the superior performance of the proposed approaches.

show abstract

“…Choosing a larger c will result in a greater AES, thus a shorter duration, as the longer experiments incurs higher costs. The impact of c has been studied in Figure 4 in Wan et al (2023).…”

Section: Utility Theory: Bayesian Optimal Effect Sizementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Effect Size Estimation for Duration Recommendation in Online Experiments: Leveraging Hierarchical Models and Objective Utility Approaches

Liu,

Wan,

McQueen

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Many practical control problems [e.g. path planning (Ramos et al , 2020), temperature control (Shokri Kalandaragh and Sabahi, 2023), clinical treatment(Wan et al , 2021; Shuvo et al , 2022), robot control (Chen et al , 2022; Zhao et al , 2023), etc.] need to consider multiple evaluation metrics and these metrics are often in conflict, i.e.…”

Section: Introductionmentioning

confidence: 99%

Multi-objective reinforcement learning based on nonlinear scalarization and long-short-term optimization

Wang

2024

RIA

View full text Add to dashboard Cite

Purpose Many practical control problems require achieving multiple objectives, and these objectives often conflict with each other. The existing multi-objective evolutionary reinforcement learning algorithms cannot achieve good search results when solving such problems. It is necessary to design a new multi-objective evolutionary reinforcement learning algorithm with a stronger searchability. Design/methodology/approach The multi-objective reinforcement learning algorithm proposed in this paper is based on the evolutionary computation framework. In each generation, this study uses the long-short-term selection method to select parent policies. The long-term selection is based on the improvement of policy along the predefined optimization direction in the previous generation. The short-term selection uses a prediction model to predict the optimization direction that may have the greatest improvement on overall population performance. In the evolutionary stage, the penalty-based nonlinear scalarization method is used to scalarize the multi-dimensional advantage functions, and the nonlinear multi-objective policy gradient is designed to optimize the parent policies along the predefined directions. Findings The penalty-based nonlinear scalarization method can force policies to improve along the predefined optimization directions. The long-short-term optimization method can alleviate the exploration-exploitation problem, enabling the algorithm to explore unknown regions while ensuring that potential policies are fully optimized. The combination of these designs can effectively improve the performance of the final population. Originality/value A multi-objective evolutionary reinforcement learning algorithm with stronger searchability has been proposed. This algorithm can find a Pareto policy set with better convergence, diversity and density.

show abstract

“…Models for dynamics and control of the disease, prioritizing hospital care, and setting policy [11][12][13][14][15][16][17][18] require information about the health status of the population. This is determined by testing for infection, which thus emerged as a crucial component of managing health policy during the current pandemic and will probably be key in future pandemics [19].…”

Section: Introductionmentioning

confidence: 99%

Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections

Mangel

2023

PLoS ONE

View full text Add to dashboard Cite

Testing remains a key tool for managing health care and making health policy during the coronavirus pandemic, and it will probably be important in future pandemics. Because of false negative and false positive tests, the observed fraction of positive tests—the surface positivity—is generally different from the fraction of infected individuals (the incidence rate of the disease). In this paper a previous method for translating surface positivity to a point estimate for incidence rate, then to an appropriate range of values for the incidence rate consistent with the model and data (the test range), and finally to the risk (the probability of including one infected individual) associated with groups of different sizes is illustrated. The method is then extended to include asymptomatic infections. To do so, the process of testing is modeled using both analysis and Monte Carlo simulation. Doing so shows that it is possible to determine point estimates for the fraction of infected and symptomatic individuals, the fraction of uninfected and symptomatic individuals, and the ratio of infected asymptomatic individuals to infected symptomatic individuals. Inclusion of symptom status generalizes the test range from an interval to a region in the plane determined by the incidence rate and the ratio of asymptomatic to symptomatic infections; likelihood methods can be used to determine the contour of the rest region. Points on this contour can be used to compute the risk (defined as the probability of including one asymptomatic infected individual) in groups of different sizes. These results have operational implications that include: positivity rate is not incidence rate; symptom status at testing can provide valuable information about asymptomatic infections; collecting information on time since putative virus exposure at testing is valuable for determining point estimates and test ranges; risk is a graded (rather than binary) function of group size; and because the information provided by testing becomes more accurate with more tests but at a decreasing rate, it is possible to over-test fixed spatial regions. The paper concludes with limitations of the method and directions for future work.

show abstract

Multi-Objective Model-based Reinforcement Learning for Infectious Disease Control

Cited by 18 publications

References 29 publications

Effect Size Estimation for Duration Recommendation in Online Experiments: Leveraging Hierarchical Models and Objective Utility Approaches

Effect Size Estimation for Duration Recommendation in Online Experiments: Leveraging Hierarchical Models and Objective Utility Approaches

Multi-objective reinforcement learning based on nonlinear scalarization and long-short-term optimization

Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections

Contact Info

Product

Resources

About