State-Dependent Resource Harvesting with Lagged Information about System States

Abstract

Markov decision processes (MDPs), which involve a temporal sequence of actions conditioned on the state of the managed system, are increasingly being applied in natural resource management. This study focuses on the modification of a traditional MDP to account for those cases in which an action must be chosen after a significant time lag in observing system state, but just prior to a new observation. In order to calculate an optimal decision policy under these conditions, possible actions must be conditioned on the previous observed system state and action taken. We show how to solve these problems when the state transition structure is known and when it is uncertain. Our focus is on the latter case, and we show how actions must be conditioned not only on the previous system state and action, but on the probabilities associated with alternative models of system dynamics. To demonstrate this framework, we calculated and simulated optimal, adaptive policies for MDPs with lagged states for the problem of deciding annual harvest regulations for mallards (Anas platyrhynchos) in the United States. In this particular example, changes in harvest policy induced by the use of lagged information about system state were sufficient to maintain expected management performance (e.g. population size, harvest) even in the face of an uncertain system state at the time of a decision.

Frequencies of decision making and monitoring in adaptive resource management

“…The overall effect of this cadence is to require averaging based on prior year status each time a new decision is made. Yet another variation involves decision making prior to monitoring each year, so that the monitoring results are not available to inform the selection of actions for that year (Johnson et al [ 16 ]). Under these conditions, possible actions must again be conditioned on the previous system and model state and the action previously taken.…”

“…Thus, while learning accrues annually, decisions are based on system state only every fourth year. In the United States, the regulations setting process for duck harvest has recently been modified so that current system state is not known (monitored) at the time a decision must be made (Johnson et al [ 16 ]). Decisions must therefore be conditioned on the previous system state and regulatory action.…”

Towards Building a Sustainable Future: Positioning Ecological Modelling for Impact in Ecosystems Management

“…Fish and Wildlife Service, Johnson et al. ( 2016 ) explained the framework used to manage mallards in the USA and Canada. Even if all of these recommendations and suggestions are followed, there is no guarantee that any particular research activity will have the desired influence on management and policy, or that it will have any impact at all.…”

“…Fish and Wildlife Service, Johnson et al. ( 2016 ) explained the framework used to manage mallards in the USA and Canada.…”

Robust stochastic control modeling of dam discharge to suppress overgrowth of downstream harmful algae

“…Abdel‐Hameed and Nakhi mathematically analyzed a tractable diffusion process model of reservoir dynamics from a long‐term perspective. Johnson et al presented a discrete stochastic process model for resource harvesting subject to information lags. Unami et al established a stochastic control model for rainwater harvesting reservoir operation during dry spells.…”