Abstract. The main goals of a wind resource assessment (WRA) at a given site are to estimate the wind speed and annual energy production (AEP) of the planned wind turbines. Several steps are involved in going from initial wind speed estimations of specific locations to a comprehensive full-scale AEP assessment. These steps differ significantly between the chosen tool and the individuals performing the examination. The goal of this work is to compare different WRA simulation tools at the Perdigão site in Portugal, for which a large amount of wind measurement data is available, in terms of both accuracy and costs. Results from nine different simulations from five different modellers were obtained via the "IEA Wind Task 31 Comparison metrics simulation challenge for wind resource assessment in complex terrain", consisting of a range of linear models, Reynolds-Averaged Navier-Stokes (RANS) computational fluid dynamics models and Large Eddy Simulations (LES). The wind speed and AEP prediction errors for three different met mast positions across the site were investigated and further translated into relative “skill” and “cost” scores, using a method previously developed by the authors. This allowed the most optimal simulation tool in terms of accuracy and cost to be chosen for this site. It was found that the RANS simulations achieved very high prediction accuracy at relatively low costs for both wind speed and AEP estimations. The LES simulations achieved great wind speed prediction for certain conditions, but at a much higher cost, which in turn also reduced the number of possible simulations, leading to a decrease in AEP prediction accuracy. For some of the simulations, the forest canopy was explicitly modelled, which was proven to be beneficial for wind speed predictions at lower heights above the ground, but lead to under-estimations of wind speeds at upper heights, decreasing the AEP prediction accuracy. Lastly, low correlation qualities between wind speed and AEP prediction error were found for each position, showing that accurate wind modelling is not necessarily the only important variable in the WRA process, and that all the steps must be considered.