Reinforcement learning in optimizing forest management

Abstract

We solve a stochastic high-dimensional optimal harvesting problem by reinforcement learning algorithms developed for agents who learn an optimal policy in a sequential decision process through repeated experience. This approach produces optimal solutions without discretization of state and control variables. Our stand-level model includes mixed species, tree size structure, optimal harvest timing, choice between rotation and continuous cover forestry, stochasticity in stand growth, and stochasticity in the occurrence of natural disasters. The optimal solution or policy maps the system state to the set of actions, i.e. clear-cut/thinning/no harvest decisions and the intensity of thinning over tree species and size classes. The algorithm repeats the solutions for deterministic problems computed earlier with time-consuming methods. Optimal policy describes harvesting choices from any initial state and reveals how the initial thinning vs. clear-cut choice depends on the economic and ecological factors. Stochasticity in stand growth increases the diversity of species composition. Despite the high variability in natural regeneration, the optimal policy closely satisfies the certainty equivalence principle. The effect of natural disasters is similar to an increase in the interest rate, but in contrast to earlier results, this tends to change the management regime from rotation forestry to continuous cover management.

Optimal Carbon Storage in Mixed-Species Size-Structured Forests

“…Stochastic phenomena with high relevance for timber production and carbon storage include fire hazard and pest damage, and as both of these are linked to the structure and quantity of living and dead trees in the stand, a model extension towards that direction would be interesting. These questions are recently studied in Malo et al (2021) but without carbon pricing. Also beyond the scope of this study are questions related to transaction costs e.g.…”

Optimizing High Dimensional Stochastic Forestry Viareinforcement Learning

“…This leads to asking how well the reduced model with a sparse state structure represents the original problem under study. Malo et al (2021) apply a size-structured matrix model with twelve pre-specified size classes, four tree species, and a continuous number of trees in each size class. Their matrix model is a direct simplification from a more detailed individualtree model obtained by a well-known transformation ( Getz and Haight, 1989 ).…”

“…While the original PPO algorithm treats all actions as continuous, the algorithm can easily be modified to handle a mixture of discrete and continuous actions, where the continuous actions are viewed as parameters of the discrete actions ( Fan et al, 2019 ). Therefore, similar to Malo et al (2021) , we approach the problem of continuous action and state spaces using the notion of parameterized action spaces, where the choice between thinning, clear-cutting, and doing nothing is modeled as discrete actions and the actual harvesting quantities are modeled as their continuous parameters. Further details on the algorithm are discussed in Appendix D .…”

“…When the individual-tree model is applied as the matrix model, the right-hand side of (A.2) is divided by the size class width, implying that the modified (A.2) determines the transition rate ( Liang and Picard, 2013 ). The matrix model we use in comparison is based on a 50-mm size class depth suggested by Bollandsås et al (2008) and applied in many studies such as Malo et al (2021) .…”

Closer-to-Nature Forest Management

“…found reduced vulnerability of uneven-aged forests to natural disturbance. In a recent modelling study,Malo et al (2021) have demonstrated the economic advantage of a structurally diverse continuous cover forest to cope with natural disturbances in boreal spruce-pine forests Knoke et al (2021b). showed faster recovery of economic value of structurally heterogeneous forests, underlining their high resilience potential after high severity disturbance.…”