Instability, Sensitivity, and Degeneracy of Discrete Exponential Families

Abstract

In applications to dependent data, first and foremost relational data, a number of discrete exponential family models has turned out to be near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We introduce the notion of instability with an eye to characterize, detect, and penalize discrete exponential family models that are near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We show that unstable discrete exponential family models are characterized by excessive sensitivity and near-degeneracy. In special cases, the subset of the natural parameter space corresponding to non-degenerate distributions and mean-value parameters far from the boundary of the mean-value parameter space turns out to be a lower-dimensional subspace of the natural parameter space. These characteristics of unstable discrete exponential family models tend to obstruct Markov chain Monte Carlo simulation and statistical inference. In applications to relational data, we show that discrete exponential family models with Markov dependence tend to be unstable and that the parameter space of some curved exponential families contains unstable subsets.

EpiModel: An R Package for Mathematical Modeling of Infectious Disease over Networks

“…The distinction is based on whether the status of one dyad influences the status of others. Dyad dependence should be modeled with care, as the patterns it induces can cascade through the network in unexpected ways, and poorly specified models can often lead to an outcome known as “model degeneracy” (Handcock, Robins, Snijders, Moody, and Besag 2003; Schweinberger 2011). With networks of any non-trivial size, κ ( θ ) may not be calculated analytically, and therefore simulation-based estimation procedures (MCMC) are used.…”

hergm: Hierarchical Exponential-Family Random Graph Models

“…Two empirical examples are presented in Sections 8.2 and 8.3. Theoretical results can be found in Handcock (2003), Schweinberger (2011), and Chatterjee and Diaconis (2013). Models with global dependence are not useful, because many real-world networks do not resemble graphs that have almost no edges or almost all possible edges.…”

“…These informal statements can be backed up by formal results: Schweinberger and Stewart (2017) proved that a wide range of ERGMs with local dependence are non-degenerate as long as the sizes of neighborhoods are either bounded above or grow slowly with the number of neighborhoods. In contrast, many ERGMs with global dependence are known to be near-degenerate (e.g., Handcock 2003;Schweinberger 2011;Chatterjee and Diaconis 2013). While some ERGMs with global dependence, such as ERGMs with geometrically weighted model terms (Snijders et al 2006;Hunter and Handcock 2006), have turned out to be useful in applications, there are no mathematical results that show whether, and under which conditions, such ERGMs are non-degenerate as the size of the network increases.…”

“…A global triangle term counts the total number of triangles in the network and induces global dependence, whereas a local triangle term counts the total number of triangles within neighborhoods and induces local dependence. ERGMs with global triangle terms are known to be near-degenerate (e.g., Handcock 2003;Schweinberger 2011;Chatterjee and Diaconis 2013). An alternative to ERGMs with global triangle terms are HERGMs with local triangle terms, which can be specified as follows:…”

“…We focus here on transitivity as the main example of network dependence, because it is one of the most interesting dependencies and one of the most challenging dependencies owing to the fact that it induces model degeneracy (e.g., Handcock 2003;Schweinberger 2011;Chatterjee and Diaconis 2013). In other words, we focus on transitivity because we want to demonstrate that HERGMs can alleviate the problem of model degeneracy in one of the most challenging cases, which suggests that HERGMs should be able to alleviate model degeneracy in all other cases as well.…”