Rating experiments in forestry: How much agreement is there in tree marking?

Abstract

The process of selecting individual trees by humans for forest management purposes is the result of a plethora of factors and processes that are hard to disentangle. And yet in the past many textbooks and other publications have maintained that this selection leads to somewhat unanimous results. In this study, we analysed the data of 36 so-called marteloscope experiments from all over Britain, which are managed by the Ae Training Centre (Scotland, UK). Our objective was (1) to establish how much agreement there actually was when asking test persons (raters) to apply two different thinning methods, low and crown thinning. In addition we (2) were interested in understanding some of the processes leading to certain levels of agreement and in relationships between the agreement measures and characteristics of forest structure. Our analysis was based on multivariate statistics, particularly using Fleiss’ kappa. This was the first time that an analysis of rater behaviour was performed at such a large scale and it revealed that the general agreement in tree selection in Britain was only slight to fair, i.e. much lower than in medical experiments. The variability of selecting individual trees was considerable. We also found that agreement in tree selection was much stronger in low-thinning as opposed to crown-thinning experiments. As the latter is an important method of Continuous Cover Forestry and British forestry is increasingly adopting this forest management type, our results suggested that there is a need to provide more training. Interestingly the different levels of agreement as identified by Fleiss’ kappa could not be explained by measures of forest structure, however, the mean conformity number, a surrogate of Fleiss’ kappa, showed correlations and indicated that conformity increased with increasing complexity of tree stem diameter structure.

Same Viewpoint Different Perspectives—A Comparison of Expert Ratings with a TLS Derived Forest Stand Structural Complexity Index

“…On the contrary, evaluations by humans are always subjective and reflect the subject's specific perspectives. Recent studies indicate that several factors, like education, or familiar management techniques and economic goals are inconsistently influencing tree selection practices, which may also largely depend on individual preferences [19,37,38]. Our finding that greener forests are perceived as richer in structure underlines the hypothesis that even forest experts unconsciously include a variety of factors in their rating of forest structures that may have no major relevance to it.…”

Resolving the trade-off between production and biodiversity conservation in integrated forest management: comparing tree selection practices of foresters and conservationists

“…Only recently studies emerged, indicating i.a. that several factors related to individual's expertise can be important for tree selection (Cosyns et al 2018;Pommerening et al 2018;Spinelli et al 2016;Vitkova et al 2016). Correspondingly, empirical implementation research points to the crucial role of forest managers for the success of integrated nature conservation policies.…”

“…While tree selection in continuous cover forestry is certainly dependent on individual experience and expertise, also professional education, socialization and habitual professional knowledge influence the decision-making processes (Maier and Winkel 2017;Spinelli et al 2016). So far, studies of tree-marking have compared subjects with rather similar professional background (foresters, forestry trainers, loggers, agronomists) (Pommerening et al 2018;Spinelli et al 2016;Vitkova et al 2016). Also, they only consider outcomes related to wood production and are based on a purely quantitative research design by looking at the outcomes of the decisions exclusively, without considering individuals' justifications more in detail.…”

“…Analysing both quantitative data on tree characteristics and qualitative justifications of tree selection decisions together provides novel insights into how which trees are selected by different relevant professional groups. By doing so, this article goes beyond existing studies on tree selection which solely rely on quantitative data (Cosyns et al 2018;Pommerening et al 2018;Spinelli et al 2016;Vitkova et al 2016), and aims to also provide insights into the motivations for tree selection. The latter is specifically relevant for conservation policy to guide practical implementation of integrated forest management strategies.…”

Decision Making in Tree Selection – Contemplating Conflicting Goals via Marteloscope Exercises

“…Although tree selection has a big influence on a forest stand's structural diversity, only recently have studies on tree-selection behaviour emerged, showing the relevance of several factors related to individual expertise and preference (Spinelli et al, 2016;Vitkova et al, 2016;Cosyns et al, 2018;Pommerening et al, 2018). Vítková et al (2016) investigate tree marking in an experimental plot before and after specific silvicultural training.…”