Management of a stage‐structured insect pest: an application of approximate optimization

Abstract

Ecological decision problems frequently require the optimization of a sequence of actions over time where actions may have both immediate and downstream effects. Dynamic programming can solve such problems only if the dimensionality is sufficiently low. Approximate dynamic programming (ADP) provides a suite of methods applicable to problems of arbitrary complexity at the expense of guaranteed optimality. The most easily generalized method is the look-ahead policy: a brute-force algorithm that identifies reasonable actions by constructing and solving a series of temporally truncated approximations of the full problem over a defined planning horizon. We develop and apply this approach to a pest management problem inspired by the Mediterranean fruit fly, Ceratitis capitata. The model aims to minimize the cumulative costs of management actions and medfly-induced losses over a single 16-week season. The medfly population is stage-structured and grows continuously while management decisions are made at discrete, weekly intervals. For each week, the model chooses between inaction, insecticide application, or one of six sterile insect release ratios. Look-ahead policy performance is evaluated over a range of planning horizons, two levels of crop susceptibility to medfly and three levels of pesticide persistence. In all cases, the actions proposed by the look-ahead policy are contrasted to those of a myopic policy that minimizes costs over only the current week. We find that look-ahead policies always out-performed a myopic policy and decision quality is sensitive to the temporal distribution of costs relative to the planning horizon: it is beneficial to extend the planning horizon when it excludes pertinent costs. However, longer planning horizons may reduce decision quality when major costs are resolved imminently. ADP methods such as the look-ahead-policy-based approach developed here render questions intractable to dynamic programming amenable to inference but should be applied carefully as their flexibility comes at the expense of guaranteed optimality. However, given the complexity of many ecological management problems, the capacity to propose a strategy that is "good enough" using a more representative problem formulation may be preferable to an optimal strategy derived from a simplified model.

Optimal control for disease vector management in SIT models: an integrodifference equation approach

“…Here we show that continuous releases predict that complete eradication of the wild population is not an optimal solution. Furthermore, integrating different control options (insecticide knockdowns, pulse or continuous SIT) to achieve cost-efficient control strategies needs more thorough investigation (Hackett and Bonsall 2018 ). Additionally, we argue that this discrete-time, continuous-space model approach is better than previous ones (described in Introduction) in finding the time and intensity of control, as it can include a variety of dispersal behaviour.…”

Insect pest control, approximate dynamic programming, and the management of the evolution of resistance

“…However, by solving Eq. 16 for a planning horizon of sufficient length to capture important behaviors, actions can be suggested that will outperform those recommended by a purely myopic policy that minimizes only the immediate costs of an action (Nicol et al 2010, Powell 2011, 2014, Powell and Meisel 2016, Hackett and Bonsall 2018. Actions taken within the look-ahead model are effectively exploratory, serving only to improve the quality of the current decision.…”

“…The integration of distinct control options for the management of an insect pest population can be conceptualized as a dynamic program, a sequential problem where decisions are made recurrently and actions implemented now can influence the options and rewards available in the future (Powell , Hackett and Bonsall ). For a given objective and set of actions, the iterative algorithmic technique of dynamic programming will identify the optimal mapping between possible states and actions.…”

“…While a comparatively brute‐force method, look‐ahead policies have several extremely valuable features: their runtime is independent of the magnitude of the state space, they are indifferent to non‐stationary parameters and they can be applied to questions where accurately approximating the downstream value of an action is problematic (Nicol et al. , Powell , , Powell and Meisel , Hackett and Bonsall ).…”

“…However, where the scale of the full problem renders it intractable, it may still be possible to identify effective actions by employing a look-ahead policy with a sufficiently long planning horizon. While a comparatively brute-force method, look-ahead policies have several extremely valuable features: their runtime is independent of the magnitude of the state space, they are indifferent to non-stationary parameters and they can be applied to questions where accurately approximating the downstream value of an action is problematic (Nicol et al 2010, Powell 2011, 2014, Powell and Meisel 2016, Hackett and Bonsall 2018.…”