Variance-Based Sensitivity Analysis: Preliminary Results in COAMPS

Marzban, Caren; Sandgathe, Scott; Doyle, James D.; Lederer, Nicholas C.

doi:10.1175/mwr-d-13-00195.1

Cited by 8 publications

(17 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All of these findings are consistent with those found for convective precipitation in Marzban et al (2014) in which a variance-based sensitivity was performed without any clustering at all. This consistency adds justification to the local and/or regression-based SA adopted here, i.e., Eq.…”

Section: Resultssupporting

confidence: 89%

“…This linear model is further justified by the results (shown below) because when it is specialized to the case of one cluster (i.e., the entire spatial domain), it reproduces the results of the variance-based approach reported in Marzban et al (2014).…”

Section: Statistical Modelsupporting

confidence: 60%

“…The strongest influence or sensitivity is from parameter 3, the fraction of available precipitation fed back to the grid from the Kain-Fritsch scheme. Increasing this fraction reduces convective precipitation and, based on the results in Marzban et al (2014), increases stable precipitation while not affecting total precipitation. It also is responsible for weakening the convective precipitation, i.e., increasing the number of weak systems.…”

Section: Conclusion and Discussionmentioning

confidence: 91%

“…Doyle et al (2011) and Jiang and Doyle (2009) examine the effect of model parameters on mountain waves. Motivated by the work of Holt et al (2011), who studied the effect of 11 model parameters on various characteristics of the forecasts, Marzban et al (2014) used a global, variance-based SA to study the effect of the same parameters and their interactions on the mean (across the forecast domain) and the center-of-gravity of precipitation. By contrast, here the effect of the model parameters is assessed on features of objects within the forecast field.…”

Section: Datamentioning

confidence: 99%

“…The SA methodology in Marzban et al (2014) is a variancebased approach that allows one to identify linear or nonlinear relationships between the forecast quantities and the model parameters and even interactions between the model parameters. As a first approximation, however, it is sufficient to estimate only the linear (i.e., main) effects because nonlinear and interaction effects are often much smaller than main effects; see, for example, pages 192, 230, 272, 314, and 329 in Montgomery (2009) and pages 33-34 in Li et al (2006).…”

Section: Statistical Modelmentioning

confidence: 99%

See 4 more Smart Citations

On the effect of model parameters on forecast objects

Marzban

Jones

et al. 2018

Geosci. Model Dev.

Self Cite

View full text Add to dashboard Cite

Abstract. Many physics-based numerical models produce a gridded, spatial field of forecasts, e.g., a temperature "map". The field for some quantities generally consists of spatially coherent and disconnected "objects". Such objects arise in many problems, including precipitation forecasts in atmospheric models, eddy currents in ocean models, and models of forest fires. Certain features of these objects (e.g., location, size, intensity, and shape) are generally of interest. Here, a methodology is developed for assessing the impact of model parameters on the features of forecast objects. The main ingredients of the methodology include the use of (1) Latin hypercube sampling for varying the values of the model parameters, (2) statistical clustering algorithms for identifying objects, (3) multivariate multiple regression for assessing the impact of multiple model parameters on the distribution (across the forecast domain) of object features, and (4) methods for reducing the number of hypothesis tests and controlling the resulting errors. The final "output" of the methodology is a series of box plots and confidence intervals that visually display the sensitivities. The methodology is demonstrated on precipitation forecasts from a mesoscale numerical weather prediction model.

show abstract

Section: Resultssupporting

confidence: 89%

Section: Statistical Modelsupporting

confidence: 60%

Section: Conclusion and Discussionmentioning

confidence: 91%

Section: Datamentioning

confidence: 99%

Section: Statistical Modelmentioning

confidence: 99%

See 3 more Smart Citations

On the effect of model parameters on forecast objects

Marzban

Jones

et al. 2018

Geosci. Model Dev.

Self Cite

View full text Add to dashboard Cite

show abstract

A methodology for sensitivity analysis of spatial features in forecasts: the stochastic kinetic energy backscatter scheme

Marzban

Tardif

Sandgathe

et al. 2019

Meteorological Applications

Self Cite

View full text Add to dashboard Cite

Stochastic kinetic energy backscatter schemes (SKEBSs) are introduced in numerical weather forecast models to represent uncertainties related to unresolved subgrid‐scale processes. These schemes are formulated using a set of parameters that must be determined using physical knowledge and/or to obtain a desired outcome. Here, a methodology is developed for assessing the effect of four factors on spatial features of forecasts simulated by the SKEBS‐enabled Weather Research and Forecasting model. The four factors include two physically motivated SKEBS parameters (the determining amplitude of perturbations applied to stream function and potential temperature tendencies), a purely stochastic element (a seed used in generating random perturbations) and a factor reflecting daily variability. A simple threshold‐based approach for identifying coherent objects within forecast fields is employed, and the effect of the four factors on object features (e.g. number, size and intensity) is assessed. Four object types are examined: upper‐air jet streaks, low‐level jets, precipitation areas and frontal boundaries. The proposed method consists of a set of standard techniques in experimental design, based on the analysis of variance, tailored to sensitivity analysis. More specifically, a Latin square design is employed to reduce the number of model simulations necessary for performing the sensitivity analysis. Fixed effects and random effects models are employed to assess the main effects and the percentage of the total variability explained by the four factors. It is found that the two SKEBS parameters do not have an appreciable and/or statistically significant effect on any of the examined object features.

show abstract

Sensitivity analysis of the convective‐scale AROME model to physical and dynamical parameters

Wimmer

Raynaud

Descamps

et al. 2022

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

A global sensitivity analysis of the convective-scale Application of Research to Operations at Mesoscale (AROME) model is performed in order to determine the most influential parameters on the forecast of different near-surface variables.For that purpose, the Morris method is applied to 21 parameters from six different physical and dynamical parametrization schemes, over different seasons. Results highlight a set of eight parameters with a noticeable influence on most variables, in particular 10 m wind speed and precipitation forecasts. The sensitivity of parameter uncertainties is also examined on different spatio-temporal scales. A clear diurnal cycle of parameters influence is observed in summer, in close connection with the convective activity. In addition, the spatial distribution of parameters influence is mostly consistent with the underlying distribution of weather forecasts. A Sobol' sensitivity analysis, based on surrogate models, mostly confirms Morris conclusions and highlights some interactions between parameters.

show abstract

Variance-Based Sensitivity Analysis: Preliminary Results in COAMPS

Cited by 8 publications

References 32 publications

On the effect of model parameters on forecast objects

On the effect of model parameters on forecast objects

A methodology for sensitivity analysis of spatial features in forecasts: the stochastic kinetic energy backscatter scheme

Sensitivity analysis of the convective‐scale AROME model to physical and dynamical parameters

Contact Info

Product

Resources

About