How expert are ‘experts’? Comparing expert predictions and empirical data on the use of farmland restoration sites by birds

Bennett, Andrew F.; Haslem, Angie; White, Matthew D.; Hollings, Tracey; Thomson, James Robertson

doi:10.1016/j.biocon.2023.110018

Cited by 5 publications

(1 citation statement)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other studies provide recommendations for the number of experts when uncertainty may reflect a lack of knowledge that can be reduced with more samples (i.e., 5–12 experts) (Hora, 2004, Hemming et al., 2018, Bennett et al., 2023). Where ecosystem definitions are difficult to interpret and rely on expert opinion, the number of experts should vary depending on the problem at hand and its underlying variability, how experts are identified, and the total size of the population of suitable experts.…”

Section: Discussionmentioning

confidence: 99%

The importance of expert selection when identifying threatened ecosystems

Travers,

Dorrough,

Shannon

et al. 2023

Conservation Biology

View full text Add to dashboard Cite

Identifying threatened ecosystem types is fundamental to conservation and management decision‐making. When identification relies upon the judgment of suitably trained experts, decisions are vulnerable to inconsistent outcomes and can lack transparency. We elicited judgements of the occurrence of a widespread, critically endangered Australian ecosystem from a diverse pool of 83 experts. We asked: (1) How many experts are required to reliably conclude that the ecosystem is present? (2) How many experts are required to build a reliable model for predicting ecosystem presence? (3) Given expert selection can narrow the range opinions, if enough experts are selected, do selection strategies affect model predictions? (4) Does a diverse selection of experts provide better model predictions? We used power and sample size calculations with a finite population of 200 experts to calculate the number of experts required to reliably assess ecosystem presence in a theoretical scenario. We then used boosted regression trees to model expert elicitation of 122 plots based on real‐world data. For a reliable consensus (90% probability of correctly identifying presence and absence), in a relatively certain scenario (85% probability of occurrence) we found at least 17 experts were required. More experts are required when occurrence was less certain; fewer if permissible error rates are relaxed. In comparison, only ∼20 experts were required for a reliable model that could predict for a range of scenarios. Expert selection strategies changed modelled outcomes, often overpredicting presence and underestimate uncertainty. However, smaller but diverse pools of experts produced similar outcomes to a model built from all contributing experts. Combining elicited judgements from a diverse pool of experts in a model‐based decision support tool provides an efficient aggregation of a broad range of expertise. Such models can improve the transparency and consistency of conservation and management decision‐making especially when ecosystems are defined by complex criteria.This article is protected by copyright. All rights reserved

show abstract

Section: Discussionmentioning

confidence: 99%

The importance of expert selection when identifying threatened ecosystems

Travers,

Dorrough,

Shannon

et al. 2023

Conservation Biology

View full text Add to dashboard Cite

show abstract

The ethical implications of Chatbot developments for conservation expertise

Sworna,

Urzedo,

Hoskins

et al. 2024

AI Ethics

View full text Add to dashboard Cite

Chatbots have emerged as a potent artificial intelligence (AI) tool for expediting expert knowledge, including evidence used for conservation research and practices. While digital technologies can support the curation and analysis of vast amounts of conservation datasets to inform best practices, AI-driven solutions raise ethical concerns around what source of evidence is used or not. This paper examines the ethical issues around sources, biases, and representation of conservation evidence formulated by chatbots. We interviewed two versions of ChatGPT, GPT-3.5-turbo and GPT-4, regarding knowledge available for ecological restoration and analysed 40,000 answers. Our results show that these chatbot developments are expanding the inclusion of diverse data sources and improving the accuracy of the responses. However, these technical developments do not necessarily imply ethical considerations in terms of fair representation and unbiased inclusion of diverse knowledge offered by different sources of expertise. While the updated model expands the descriptions ofgeographical locations and organizations, there remain limitations regarding equitable representation of different expertise and stakeholders. The updated version of GPT still relies heavily on evidence from high-income countries (88%), North American expertise (67%), and male academics (46%) with limited contributions from minority groups, such as Indigenous organizations (10%) and low-income countries (2%). In conclusion, the ethical implications within generative AI reveal the crucial requirement of human-centered negotiations to consider how knowledge practices are legitimized and embedded in the development and use of chatbots.

show abstract

Evaluating approaches for integrating species distributions in spatial conservation planning

Winiarski,

Shipley,

Fowler

et al. 2024

Conservat Sci and Prac

View full text Add to dashboard Cite

Map‐based decision support tools (DSTs) that use species distributions are an important means of identifying priority areas for conservation. The Wisconsin Waterfowl Habitat Conservation Strategy (WWHCS) uses a DST to identify priority ecological landscapes and watersheds to guide waterfowl habitat projects. The WWHCS DST relies on waterfowl habitat suitability layers derived through expert opinion in lieu of species distributions, a common approach in DSTs. Given the subjectivity of expert opinion, model‐driven species distributions such as those available from community science projects could provide more reliable information and better identify areas for waterfowl conservation. Here, we explore the application of relative abundance products available through the eBird Status and Trends project as an alternative to expert‐derived habitat suitability layers in the WWHCS DST. Our objectives were to compare seasonal species distributions from habitat suitability models (expert‐derived) and species distribution models (eBird‐derived) and determine whether differences influenced DST prioritizations. Correlations between expert‐ and eBird‐derived distributions were generally low to moderate for breeding and fall layers (ρ: −0.03–0.76), and lowest for spring (ρ: −0.49–0.72). There was also minimal agreement among top‐ranked ecological landscapes (40%) and watersheds (28%) between the two versions of the DST. Finally, we compare tradeoffs and suggest a model‐driven approach for the WWHCS DST. However, additional work validating eBird relative abundance against professional surveys and empirical studies evaluating waterfowl habitat selection and vital rates are important future considerations for the DST and waterfowl habitat conservation in Wisconsin.

show abstract

How expert are ‘experts’? Comparing expert predictions and empirical data on the use of farmland restoration sites by birds

Cited by 5 publications

References 40 publications

The importance of expert selection when identifying threatened ecosystems

The importance of expert selection when identifying threatened ecosystems

The ethical implications of Chatbot developments for conservation expertise

Evaluating approaches for integrating species distributions in spatial conservation planning

Contact Info

Product

Resources

About