Complex decisions made simple: a primer on stochastic dynamic programming

Marescot, Lucile; Chapron, Guillaume; Chadès, Iadine; Fackler, Paul L.; Duchamp, Christophe; Marboutin, Éric; Giménez, Olivier

doi:10.1111/2041-210x.12082

Cited by 122 publications

(136 citation statements)

References 66 publications

Supporting

Mentioning

133

Contrasting

Order By: Relevance

“…In a similar fashion, one may solve infinite horizon problems using the method of value iteration, which is analogous to backwards induction applied repeatedly from a zero terminal rewards function ϕ ( x ) = 0 for all x , until some convergence criterion for f ( x , t ) is reached (see Marescot et al, for an overview). We compare results obtained using these numerical methods with the proposed matrix methods.…”

Section: Methodsmentioning

confidence: 99%

Matrix methods for stochastic dynamic programming in ecology and evolutionary biology

et al. 2019

View full text Add to dashboard Cite

Organisms are constantly making tradeoffs. These tradeoffs may be behavioural (e.g. whether to focus on foraging or predator avoidance) or physiological (e.g. whether to allocate energy to reproduction or growth). Similarly, wildlife and fishery managers must make tradeoffs while striving for conservation or economic goals (e.g. costs vs. rewards). Stochastic dynamic programming (SDP) provides a powerful and flexible framework within which to explore these tradeoffs. A rich body of mathematical results on SDP exist but have received little attention in ecology and evolution. Using directed graphs – an intuitive visual model representation – we reformulated SDP models into matrix form. We synthesized relevant existing theoretical results which we then applied to two canonical SDP models in ecology and evolution. We applied these matrix methods to a simple illustrative patch choice example and an existing SDP model of parasitoid wasp behaviour. The proposed analytical matrix methods provide the same results as standard numerical methods as well as additional insights into the nature and quantity of other, nearly optimal, strategies, which we may also expect to observe in nature. The mathematical results highlighted in this work also explain qualitative aspects of model convergence. An added benefit of the proposed matrix notation is the resulting ease of implementation of Markov chain analysis (an exact solution for the realized states of an individual) rather than Monte Carlo simulations (the standard, approximate method). It also provides an independent validation method for other numerical methods, even in applications focused on short‐term, non‐stationary dynamics. These methods are useful for obtaining, interpreting, and further analysing model convergence to the optimal time‐independent (i.e. stationary) decisions predicted by an SDP model. SDP is a powerful tool both for theoretical and applied ecology, and an understanding of the mathematical structure underlying SDP models can increase our ability to apply and interpret these models.

show abstract

Section: Methodsmentioning

confidence: 99%

Matrix methods for stochastic dynamic programming in ecology and evolutionary biology

et al. 2019

View full text Add to dashboard Cite

show abstract

“…SDP can be applied where sequential management decisions are made to stochastic systems with a finite number of states (Bellman , Lubow , Mangel and Clark , Marescot et al. ). After discretizing the system into states, the first step of an SDP is to define a management objective.…”

Section: Methodsmentioning

confidence: 99%

Learning about colonization when managing metapopulations under an adaptive management framework

Southwell

Hauser

McCarthy

2016

Ecological Applications

View full text Add to dashboard Cite

Adaptive management is a framework for resolving key uncertainties while managing complex ecological systems. Its use has been prominent in fisheries research and wildlife harvesting; however, its application to other areas of environmental management remains somewhat limited. Indeed, adaptive management has not been used to guide and inform metapopulation restoration, despite considerable uncertainty surrounding such actions. In this study, we determined how best to learn about the colonization rate when managing metapopulations under an adaptive management framework. We developed a mainland-island metapopulation model based on the threatened bay checkerspot butterfly (Euphydryas editha bayensis) and assessed three management approaches: adding new patches, adding area to existing patches, and doing nothing. Using stochastic dynamic programming, we found the optimal passive and active adaptive management strategies by monitoring colonization of vacant patches. Under a passive adaptive strategy, increasing patch area was best when the expected colonization rate was below a threshold; otherwise, adding new patches was optimal. Under an active adaptive strategy, it was best to add patches only when we were reasonably confident that the colonization rate was high. This research provides a framework for managing mainland-island metapopulations in the face of uncertainty while learning about the dynamics of these complex systems.

show abstract

“…We showed that SDM can be used to identify appropriate target harvest rates in the absence of information needed to determine optimal state-dependent decisions (e.g., Martin et al 2009, Marescot et al 2013 or employ robust statedependent harvest control rules (e.g., Punt 2006, Deroba and Bence 2008, Hilborn 2012. We showed that SDM can be used to identify appropriate target harvest rates in the absence of information needed to determine optimal state-dependent decisions (e.g., Martin et al 2009, Marescot et al 2013 or employ robust statedependent harvest control rules (e.g., Punt 2006, Deroba and Bence 2008, Hilborn 2012.…”

Section: Identifying Reference Points For Assessment-limited Populationsmentioning

confidence: 99%

“…AHM employs dynamic decision analyses to identify optimal harvest strategies recurrently over time, and reduces uncertainty through targeted monitoring programs that provide rigorous estimates of important state variables (e.g., abundance) and the responses of those variables to management (Nichols et al 1995, Johnson et al 1997. Proponents of AHM emphasize use of dynamic optimization methods (e.g., Lubow 1996, Marescot et al 2013 to identify optimal state-dependent policies at regular intervals over time as a function of population abundance and environmental conditions (Johnson et al 1997, Martin et al 2009. Although theoretically optimal, implementation of these methods presupposes a formal monitoring and assessment program is in place to estimate abundance or provide reliable, unbiased indices of abundance at regular intervals so that optimal policies can be updated over time.…”

Section: Introductionmentioning

confidence: 99%

Identifying target reference points for harvesting assessment‐limited wildlife populations: a case study

Stevens

Bence

Porter

et al. 2017

Ecological Applications

View full text Add to dashboard Cite

Identifying appropriate strategies for sustainable harvest is a challenge for many terrestrial vertebrate species because of uncertain system dynamics, limited data to inform population models, and potentially conflicting objectives that seek to harvest and maintain populations at desirable levels. The absence of monitoring and assessment infrastructure needed to regularly estimate abundance accentuates this challenge for many species, and limits application of rigorous state-dependent frameworks for decision making that are commonly advocated in natural resource management. Reference points, which define management targets or triggers for changing management, are often used to guide decision-making, but suffer from ambiguity when developed without explicit consideration of uncertainty or trade-offs among competing objectives. We describe an approach for developing unambiguous target reference points for assessment-limited species using structured decision making, and demonstrate the approach to develop target harvest rates for management of fall Wild Turkey (Meleagris gallopavo) harvests in the face of uncertain population and harvest dynamics. We use simulation and decision analyses to identify harvest rates that are optimal for accomplishing explicit management objectives in the face of uncertainty, and harvest rates with robust performance over broad regions of the demographic and harvest model parameter space. We demonstrate that population and harvest parameters commonly uncertain to wildlife managers interact to determine appropriate target harvest rates for Wild Turkeys, and that formally acknowledging a range of plausible values for structurally uncertain parameters results in more conservative target reference points than suggested by previously published studies. The structured decision making framework described here provides a natural conceptual and quantitative framework for extending our approach to develop unambiguous harvest targets for other assessment-limited wildlife populations while formally acknowledging structural uncertainty in system dynamics.

show abstract

Complex decisions made simple: a primer on stochastic dynamic programming

Cited by 122 publications

References 66 publications

Matrix methods for stochastic dynamic programming in ecology and evolutionary biology

Matrix methods for stochastic dynamic programming in ecology and evolutionary biology

Learning about colonization when managing metapopulations under an adaptive management framework

Identifying target reference points for harvesting assessment‐limited wildlife populations: a case study

Contact Info

Product

Resources

About