Mapping Uncertainty Due to Missing Data in the Global Ocean Health Index

Abstract

Indicators are increasingly used to measure environmental systems; however, they are often criticized for failing to measure and describe uncertainty. Uncertainty is particularly difficult to evaluate and communicate in the case of composite indicators which aggregate many indicators of ecosystem condition. One of the ongoing goals of the Ocean Health Index (OHI) has been to improve our approach to dealing with missing data, which is a major source of uncertainty. Here we: (1) quantify the potential influence of gapfilled data on index scores from the 2015 global OHI assessment; (2) develop effective methods of tracking, quantifying, and communicating this information; and (3) provide general guidance for implementing gapfilling procedures for existing and emerging indicators, including regional OHI assessments. For the overall OHI global index score, the percent contribution of gapfilled data was relatively small (18.5%); however, it varied substantially among regions and goals. In general, smaller territorial jurisdictions and the food provision and tourism and recreation goals required the most gapfilling. We found the best approach for managing gapfilled data was to mirror the general framework used to organize, calculate, and communicate the Index data and scores. Quantifying gapfilling provides a measure of the reliability of the scores for different regions and components of an indicator. Importantly, this information highlights the importance of the underlying datasets used to calculate composite indicators and can inform and incentivize future data collection.

Our path to better science in less time using open data science tools

“…Our first priority was to code all data preparation, create a standard format for final data layers, and do so using a single programmatic language, R 42 . Code enables us to reproduce the full process of data preparation, from data download to final model inputs 37,53 , and a single language makes it more practical for our team to learn and contribute col laboratively. We code in R and use RStudio 43 to power our work flow because it has a user friendly interface and built in tools useful for coders of all skill levels, and, importantly, it can be configured with Git to directly sync with GitHub online (See 'Collaboration').…”

Quantifying uncertainty in the wild‐caught fisheries goal of the Ocean Health Index

“…The nature of composite indicators and indices make accounting for uncertainty difficult, but it is still possible (some examples provided in Frazier et al. ()). This analysis provides an approach for estimating uncertainty around OHI fisheries goal status, and the results support the rationale for making this effort.…”

“…Estimating missing data, as opposed to ignoring it, can reduce bias in indicator scores (Frazier et al., ). Eliminating the need to gapfill data for the fisheries goal would require dramatic improvements to catch data quality or, ideally, the availability of other data sources for full stock assessments.…”

“…There are many gaps in the data used to calculate fisheries status; for example, in the 2016 assessment, B/B MSY values could not be calculated for 58% of the global catch (Halpern et al., ). Consequently, missing stock scores were estimated using other means to prevent scoring bias (Frazier, Longo, & Halpern, ). In most cases, B/B MSY values could not be calculated because catch was not reported at the required species level.…”

Using the Ocean Health Index to Identify Opportunities and Challenges to Improving Southern Ocean Ecosystem Health

“…Although an approach was developed to capture risk of error associated with poor data quality, by using as proxy the degree to which a score relies on gap-filled data (i.e., indirectly derived or modeled), this is mainly suited for large scale applications of the Index, and only addresses one type of source of uncertainty (Frazier et al, 2016). Here, we try to address these concerns by thoroughly documenting the data sources and methods in the Supplement, and discussing the main caveats to the interpretation of the scores.…”