Improving estimates of population status and trend with superensemble models

Abstract

Fishery managers must often reconcile conflicting estimates of population status and trend. Superensemble models, commonly used in climate and weather forecasting, may provide an effective solution. This approach uses predictions from multiple models as covariates in an additional “superensemble” model fitted to known data. We evaluated the potential for ensemble averages and superensemble models (ensemble methods) to improve estimates of population status and trend for fisheries. We fit four widely applicable data‐limited models that estimate stock biomass relative to equilibrium biomass at maximum sustainable yield (B/BMSY). We combined these estimates of recent fishery status and trends in B/BMSY with four ensemble methods: an ensemble average and three superensembles (a linear model, a random forest and a boosted regression tree). We trained our superensembles on 5,760 simulated stocks and tested them with cross‐validation and against a global database of 249 stock assessments. Ensemble methods substantially improved estimates of population status and trend. Random forest and boosted regression trees performed the best at estimating population status: inaccuracy (median absolute proportional error) decreased from 0.42 – 0.56 to 0.32 – 0.33, rank‐order correlation between predicted and true status improved from 0.02 – 0.32 to 0.44 – 0.48 and bias (median proportional error) declined from −0.22 – 0.31 to −0.12 – 0.03. We found similar improvements when predicting trend and when applying the simulation‐trained superensembles to catch data for global fish stocks. Superensembles can optimally leverage multiple model predictions; however, they must be tested, formed from a diverse set of accurate models and built on a data set representative of the populations to which they are applied.

Predicting recruitment density dependence and intrinsic growth rate for all fishes worldwide using a data‐integrated life‐history model

“…This would allow researchers to link management decision theory to the likely population dynamics for all species worldwide. Ensemble modelling : Stock assessment will generally be more robust if management advice is based on an ensemble of alternative life‐history assumptions (Stewart & Martell, ). Results from a model ensemble can be presented using a decision table (Hilborn, Pikitch, & Francis, ) or via ensemble weighting of model results (Anderson et al, ), but either presentation requires some objective method for determining the weight of different unknown “states of nature.” I therefore recommend that stock assessments increasingly present results using an ensemble of life‐history values, where model weights can be obtained from the multivariate distribution for these parameters. Strategic decisions : Finally, strategic decision‐making in ecosystem‐based management is increasingly informed by models that include dynamics for multiple species and physical drivers (Fulton et al, ). For example, ecosystem models have been used to forecast likely impacts of ocean acidification, temperature changes or invasive species on fisheries potential (Cheung, Lam, & Pauly, ; Marshall et al, ; Morello et al, ).…”

Forecast skill for predicting distribution shifts: A retrospective experiment for marine fishes in the Eastern Bering Sea

“…Although there are sufficient data to conduct retrospective skill testing over seasonal or short‐term forecasts, there is surprisingly little research using skill testing to compare performance among alternative models. I therefore recommend skill testing to either identify which models to use for different planning horizons, or how to weight predictions from multiple models when using an ensemble model for forecasting (e.g., Anderson et al., ).…”

Applying a New Ensemble Approach to Estimating Stock Status of Marine Fisheries around the World

“…Previous analyses showed that a random forest superensemble consistently had the best or among the best performance characteristics when compared to other possible superensemble regression models (Anderson et al . ). The superensemble outperformed the individual models in cross‐validation on simulated data with, for example, a median absolute proportional error in B/B MSY of 0.32 compared to 0.42–0.56 for the individual models (Anderson et al .…”

“…The superensemble outperformed the individual models in cross‐validation on simulated data with, for example, a median absolute proportional error in B/B MSY of 0.32 compared to 0.42–0.56 for the individual models (Anderson et al . ).…”

“…A recent development in data‐limited stock assessment methods has been to combine the estimates of exploitation status, B/B MSY , where B is current stock biomass and B MSY is the biomass estimated to result in maximum sustainable yield from multiple catch‐only models in a superensemble (Anderson et al . and Supporting Information). A superensemble (Krishnamurti et al .…”