Statistical Distances and the Construction of Evidence Functions for Model Adequacy

Abstract

Over the past years, distances and divergences have been extensively used not only in the statistical literature or in probability and information theory, but also in other scientific areas such as engineering, machine learning, biomedical sciences, as well as ecology. Statistical distances, viewed either as building blocks of evidence generation or as evidence generation vehicles in themselves, provide a natural way to create a global framework for inference in parametric and semiparametric models. More precisely, quadratic distance measures play an important role in goodness-of-fit tests, estimation, prediction or model selection. Provided that specific properties are fulfilled, alternative statistical distances (or divergences) can effectively be used to construct evidence functions. In the present article, we discuss an intrinsic approach to the notion of evidence and present a brief literature review related to its interpretation. We examine several statistical distances, both quadratic and non-quadratic, and their properties in relation to important aspects of evidence generation. We provide an extensive description of their role in model identification and model assessment. Further, we introduce an explanatory plot that is based on quadratic distances to visualize the strength of evidence provided by the ratio of standardized quadratic distances and exemplify its use. In this setting, emphasis is placed on determining the sense in which we can provide meaningful interpretations of the distances as measures of statistical loss. We conclude by summarizing the main contributions of this work.

Assessing the Global and Local Uncertainty of Scientific Evidence in the Presence of Model Misspecification

“…Research into diagnostics to identify these cases is called for (Cook and Weisberg, 1982). Useful diagnostics will involve more than measures of the adequacy of single models (e.g., Markatou and Sofikitou, 2019) they must somehow include measures of the geometry of the generating process and the competing models (Dennis et al, 2019;Ponciano and Taper, 2019).…”

“…The researcher should of course recognize that not all data is equally informative and seek data that will distinguish the two models (e.g., Cooper et al, 2008). Another choice that could be made, particularly if large amounts of data have already been collected, is to decide that both models are adequate for the intended purposes (Lindsay, 2004;Markatou and Sofikitou, 2019). Intervals 7, 8, 9, and 10 are reflections of intervals 5, 4, 3, and 2, only in this case they are misleading.…”

“…Different statistical divergences could be used to construct model adequacy measures and thus evidence functions (see Lele, 2004a;Markatou and Sofikitou, 2019). Each will have its own properties, and each could be useful in different circumstances.…”

Evaluation of Remote Site Incubators to Incubate Wild‐ and Hatchery‐Origin Westslope Cutthroat Trout Embryos

“…Initial analyses indicated that these data were overdispersed, so we added a random‐level variable for observation to account for this problem (Bates et al 2015). Adding the random effect of observation (either egg lot or RSI) in these two analyses improved the adequacy of the models (Markatou and Sofikitou 2019; using Bayesian information criteria [BIC] and the difference in BIC values from the top model [ΔBIC] > 2,000), so we evaluated models that included observation as a random effect.…”

How Should We Quantify Uncertainty in Statistical Inference?

“…The law of the likelihood corresponds to using the Kullback-Leibler divergence but other measures, such as the Hellinger divergence, Jeffrey's divergence, etc. also lead to appropriate quantification of the strength of evidence with some important robustness properties (Lele, 2004;Markatou and Sofikitou, 2019).…”