Untangling Neural Nets

Abstract

B eck, King, and Zeng (2000) offer both a sweeping critique of the quantitative security studies field and a bold new direction for future research. Despite important strengths in their work, we take issue with three aspects of their research: (1) the substance of the logit model they compare to their neural network, (2) the standards they use for assessing forecasts, and (3) the theoretical and modelbuilding implications of the nonparametric approach represented by neural networks. We replicate and extend their analysis by estimating a more complete logit model and comparing it both to a neural network and to a linear discriminant analysis. Our work reveals that neural networks do not perform substantially better than either the logit or the linear discriminant estimators. Given this result, we argue that more traditional approaches should be relied upon due to their enhanced ability to test hypotheses. D uring the 1990s, quantitative security studies became an increasingly prominent and sophisticated area of inquiry within our discipline. Just in American Political Science Review, more than 20 articles over the past five years have applied some form of econometric technique to the study of international conflict. In particular, estimators based on the general linear model have been central to the development of extensive literatures on deterrence, the impact of democracy and trade on international conflict, and many other issues. In the March 2000 issue of this Review (Vol. 94, No. l, 21-36) Beck, King, and Zeng (hereafter, BKZ) offered a sweeping critique of these research programs and a bold new direction for future research. They contend that standard parametric procedures are not up to the task of estimating the causes of international conflict because these relationships are "highly non-linear, massively interactive, and heavily context dependent or contingent" (22). 1 Consequently, improvements in theory and data are for naught, without a substitute for the inadequate, inflexible models based on the general linear model. As an alternative, BKZ introduce a statistical estimator, commonly called a "neural network," that can approximate complex relationships without prior assumptions.The evidence for BKZ's claim is the alleged ability of neural networks to improve forecasts of the onset of militarized disputes. We are, however, less sanguine about the forecasting performance of neural networks compared to more traditional logit models. If the goal is to reject an entire research paradigm, it seems appropriate to use the best model that paradigm has to offer. But we contend that BKZ omitted several variables and

Assessing Different Bayesian Neural Network Models for Militarized Interstate Dispute

“…Contrary to recent critiques that emphasize interpretability issues in neural networks (de Marchi et al, 2004), the ARD method demonstrates once more that neural networks are not black boxes. Indeed the Bayesian framework offers this useful feature, introduced by MacKay (1995) and Neal (1996), to help identify important variables.…”

“…MacKay (1992aMacKay ( , 1992b has suggested a mathematically convenient Gaussian approximation method to analytically solve the integration in neural networks that conflict scholars have recently applied to conflict data (Beck et al, 2000(Beck et al, , 2004de Marchi et al, 2004). To analytically calculate the values of interest in Equation 3, MacKay approximates the posterior probability of the weights to a Gaussian distribution.…”

Learning from the Past and Stepping into the Future: Toward a New Generation of Conflict Prediction

“…Still there were a few scholars that continued to make predictions (yes, about the future), including Gurr and Harff (), Krause (), Davies and Gurr (), Pevehouse and Goldstein (), Schrodt and Gerner (), King and Zeng (), O'Brien (), Bueno de Mesquita (), Fearon and Laitin (), de Marchi, Gelpi, and Grynaviski (), Enders and Sandler (), Leblang and Satyanath (), Ward, Siverson and Cao (), Brandt, Colaresi, and Freeman (), Bennett and Stam (), and Gleditsch and Ward (), among a few others . However, just in the last few years, the field of conflict forecasting has expanded tremendously.…”

Toward a Continuous Specification of the Democracy-Autocracy Connection

“…Undoubtedly, there are other combinations out there in the large empirical literature on the democratic peace. One additional specification recently seen in print is that of de Marchi, Gelpi, and Grynaviski (2004), which includes individual democracy scores, their interaction, and the square of the interaction. Beck, King and Zeng (2004) doubt this specification (particularly the squared term), questioning whether it is in fact a standard specification and arguing that it specifies only one possible set of relationships among the democracy variables.…”