Residual analysis methods for space–time point processes with applications to earthquake forecast models in California

Abstract

Modern, powerful techniques for the residual analysis of spatialtemporal point process models are reviewed and compared. These methods are applied to California earthquake forecast models used in the Collaboratory for the Study of Earthquake Predictability (CSEP). Assessments of these earthquake forecasting models have previously been performed using simple, low-power means such as the L-test and N-test. We instead propose residual methods based on rescaling, thinning, superposition, weighted K-functions and deviance residuals. Rescaled residuals can be useful for assessing the overall fit of a model, but as with thinning and superposition, rescaling is generally impractical when the conditional intensity λ is volatile. While residual thinning and superposition may be useful for identifying spatial locations where a model fits poorly, these methods have limited power when the modeled conditional intensity assumes extremely low or high values somewhere in the observation region, and this is commonly the case for earthquake forecasting models. A recently proposed hybrid method of thinning and superposition, called superthinning, is a more powerful alternative. The weighted K-function is powerful for evaluating the degree of clustering or inhibition in a model. Competing models are also compared using pixel-based approaches, such as Pearson residuals and deviance residuals. The different residual analysis techniques are demonstrated using the CSEP models and are used to highlight certain deficiencies in the models, such as the overprediction of seismicity in inter-fault zones for the model proposed by Helmstetter, Kagan and Jackson [Seismological

Prospective and retrospective evaluation of five-year earthquake forecast models for California

“…Residuals scores from the point process modelling literature (Lawson 1993;Baddeley et al 2005;Clements et al 2011) are well-suited to resolve this particular issue. Forecasts can be represented as space-time point process models, where binwise forecasts are represented as infinitesimal seismicity rates at specific locations.…”

“…Forecasts can be represented as space-time point process models, where binwise forecasts are represented as infinitesimal seismicity rates at specific locations. By representing a forecast in such a way, one can locate specific areas where a forecast performs well or poorly through the use of point process residual methods (Clements et al 2011;Schneider et al 2014). An advantage to this representation is that no assumptions regarding binwise seismicity independence and Poisson seismicity within bins are required.…”

“…Deviance residuals are point process residual metrics which can be used to highlight differences in performance between two forecasts in spatial bins (Clements et al 2011;Schneider et al 2014). Within bins containing highly positive or negative deviance residuals, one forecast considerably outperforms the other in forecasting the observed seismicity.…”

“…This enables us to track how the presence of particular model features affects forecast performance, and how this varies over space. Clements et al (2011) evaluated the mainshock+aftershock seismicity models developed by Helmstetter et al (2007), Shen et al (2007) and Kagan et al (2007) during the evaluation period of the original RELM experiment. They found that the HELMSTETTER model outperformed the other two models within the Imperial earthquake cluster and the earthquake cluster just north of the California-Mexico border (lon ≈ 116.0 • W and lat ≈ 32.7 • N).…”

A parimutuel gambling perspective to compare probabilistic seismicity forecasts

“…Parimutuel gambling maps also reveal features that corresponding maps of likelihood cannot. [We include maps of net returns and likelihood for each forecast in the Supporting Information; see Clements et al (2011) for additional examples of graphical methods for forecast evaluation and comparison.] For example, consider the main shock+aftershock Zechar et al (2013).…”