Validation metrics for turbulent plasma transport

Abstract

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnostics to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. The utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak [J. L. Luxon, Nucl. Fusion 42, 614 (2002)], as part of a multi-year transport model validation activity.

Validation study of a drift-wave turbulence model for CSDX linear plasma device

“…7 In this approach, one identifies a single input parameter (often the driving gradient of the dominant micro-instability, or a damping mechanism), and performs a discrete set of simulations in which this parameter is systematically varied about the experimental base value, holding all other inputs in the model fixed. Here, the base case is chosen as the statistical median in the range of uncertainties.…”

“…20 Synthetic diagnostics are computational algorithms used to transform the simulation results into predicted experimentally observable quantities and enable meaningful quantitative comparisons. 7 Three main types of diagnostics that have been used in the CSDX linear device: multi-tip Langmuir probes, visible light fast imaging camera, 21 and laser induced fluorescence. 12 In this publication, we will discuss the construction of synthetic Langmuir probe diagnostics, which will be used in Sec.…”

“…Other examples include Ricci et al, [3][4][5] who studied the influence of synthetic diagnostics on Langmuir probe observations, and Russell et al, 6 who studied synthetic gas puff imaging diagnostics in the scrapeoff-layer turbulence setting. More recently, Holland 7 discussed the importance of uncertainty quantification and its inclusion within validation metrics, and the need for utilizing synthetic diagnostics for code-experiment comparisons.…”

Uncertainty propagation by using spectral methods: A practical application to a two-dimensional turbulence fluid model

“…In Fig. 4 we present the distributions of η obtained at x = 40 (first row) and x = 60 (second row) with the spectral code for (K, L, M, N ) = (1, 12, 9, 1), (2,14,11,2), (3,16,13,3) (left, middle, and right columns, respectively). We observe that the distributions slightly depend on the number of spectral terms used in the simulations and that η does not depend linearly on…”

“…In order to assess the reliability of a simulation and validate it against experimental measurements, it is essential to estimate the uncertainties affecting its numerical results [1][2][3].…”