Reproducibility in Subsurface Geoscience

Abstract

Reproducibility, the extent to which consistent results are obtained when an experiment or study is repeated, sits at the foundation of science. The aim of this process is to produce robust findings and knowledge, with reproducibility being the screening tool to benchmark how well we are implementing the scientific method. However, the re-examination of results from many disciplines has caused significant concern as to the reproducibility of published findings. This concern is well-founded—our ability to independently reproduce results build trust within the scientific community, between scientists and policy makers, and the general public. Within geoscience, discussions and practical frameworks for reproducibility are in their infancy, particularly in subsurface geoscience, an area where there are commonly significant uncertainties related to data (e.g., geographical coverage). Given the vital role of subsurface geoscience as part of sustainable development pathways and in achieving Net Zero, such as for carbon capture storage, mining, and natural hazard assessment, there is likely to be increased scrutiny on the reproducibility of geoscience results. We surveyed 346 Earth scientists from a broad section of academia, government, and industry to understand their experience and knowledge of reproducibility in the subsurface. More than 85% of respondents recognised there is a reproducibility problem in subsurface geoscience, with >90% of respondents viewing conceptual biases as having a major impact on the robustness of their findings and overall quality of their work. Access to data, undocumented methodologies, and confidentiality issues (e.g., use of proprietary data and methods) were identified as major barriers to reproducing published results. Overall, the survey results suggest a need for funding bodies, data providers, research groups, and publishers to build a framework and a set of minimum standards for increasing the reproducibility of, and political and public trust in, the results of subsurface studies.

Capturing geological uncertainty in salt cavern developments for hydrogen storage: Case study from Southern North Sea

“…The necessity for geological models to be reliable and reproducible is essential where they underpin vital developments as part of sustainable pathways and in achieving Net Zero (Steventon et al, 2022). We compare the layered evaporite salt model using seismic specific data (Sections 4.1.1, Figure 8) and using basin wide depth data (Section 4.1.3, Figure 10).…”

“…Within subsurface geosciences, practical frameworks for reproducibility are in their infancy, particularly where there are significant uncertainties related to data (Steventon et al, 2022).…”

“…The UK is currently undergoing a shift in the supply of energy to meet its 2050 net-zero obligations, with installed wind power capacity in 2023 reaching 27.9 GW (Staffell et al, 2023). For 100% renewable penetration by 2035 in the UK, Cárdenas et al (2021) found that with the optimum mix of renewable technologies and allowing for over-generation, the UK would require ~42 TWh of LDES, far lower than the suggested 115 TWh needed if no over generation is allowed.…”

Quantifying fault interpretation uncertainties and their impact on fault seal and seismic hazard analysis

“…Repeatability, which is the ability to replicate the data and interpretations of a study, is recognised as a crucial aspect of any experiment (e.g., Goodman, 2016). Geology, in particular, is susceptible to subjective uncertainty due to incomplete datasets and the lack of consensus within the research community regarding key concepts and research methods (Frodeman, 1995;Bond, 2015;Pérez-Díaz et al, 2020;Steventon et al, 2022;Magee et al, 2023;Robledo Carvajal et al, 2023). For seismic reflection datasets, subjective uncertainties can lead to multiple interpretations being drawn from the same seismic image (e.g., Bond et al, 2007;Alcalde et al, 2017).…”