Abstract-The ability to successfully predict the future behavior of a system is a strong indication that the system is well understood. Certainly many details of the earthquake system remain obscure, but several hypotheses related to earthquake occurrence and seismic hazard have been proffered, and predicting earthquake behavior is a worthy goal and demanded by society. Along these lines, one of the primary objectives of the Regional Earthquake Likelihood Models (RELM) working group was to formalize earthquake occurrence hypotheses in the form of prospective earthquake rate forecasts in California. RELM members, working in small research groups, developed more than a dozen 5-year forecasts; they also outlined a performance evaluation method and provided a conceptual description of a Testing Center in which to perform predictability experiments. Subsequently, researchers working within the Collaboratory for the Study of Earthquake Predictability (CSEP) have begun implementing Testing Centers in different locations worldwide, and the RELM predictability experiment-a truly prospective earthquake prediction effort-is underway within the U.S. branch of CSEP. The experiment, designed to compare time-invariant 5-year earthquake rate forecasts, is now approximately halfway to its completion. In this paper, we describe the models under evaluation and present, for the first time, preliminary results of this unique experiment. While these results are preliminary-the forecasts were meant for an application of 5 years-we find interesting results: most of the models are consistent with the observation and one model forecasts the distribution of earthquakes best. We discuss the observed sample of target earthquakes in the context of historical seismicity within the testing region, highlight potential pitfalls of the current tests, and suggest plans for future revisions to experiments such as this one.

Please refer to any applicable terms of use of the publisher.

No abstract

No abstract

For government officials and the public to act on real-time forecasts of earthquakes, the seismological community needs to develop confidence in the underlying scientific hypotheses of the forecast generating models by assessing their predictive skill. For this purpose, the Collaboratory for the Study of Earthquake Predictability (CSEP) provides cyberinfrastructure and computational tools to evaluate earthquake forecasts. Here, we introduce pyCSEP, a Python package to help earthquake forecast developers embed model evaluation into the model development process. The package contains the following modules: (1) earthquake catalog access and processing, (2) data models for earthquake forecasts, (3) statistical tests for evaluating earthquake forecasts, and (4) visualization routines. pyCSEP can evaluate earthquake forecasts expressed as expected rates in space-magnitude bins, and simulation-based forecasts that produce thousands of synthetic seismicity catalogs. Most importantly, pyCSEP contains community-endorsed implementations of statistical tests to evaluate earthquake forecasts, and provides well defined file formats and standards to facilitate model comparisons. The toolkit will facilitate integrating new forecasting models into testing centers, which evaluate forecast models and prediction algorithms in an automated, prospective and independent manner, forming a critical step towards reliable operational earthquake forecasting.