Sample size matters: investigating the effect of sample size on a logistic regression debris flow susceptibility model

Abstract

Abstract. Predictive spatial modelling is an important task in natural hazard assessment and regionalisation of geomorphic processes or landforms. Logistic regression is a multivariate statistical approach frequently used in predictive modelling; it can be conducted stepwise in order to select from a number of candidate independent variables those that lead to the best model. In our case study on a debris flow susceptibility model, we investigate the sensitivity of model selection and quality to different sample sizes in light of the following problem: on the one hand, a sample has to be large enough to cover the variability of geofactors within the study area, and to yield stable results; on the other hand, the sample must not be too large, because a large sample is likely to violate the assumption of independent observations due to spatial autocorrelation. Using stepwise model selection with 1000 random samples for a number of sample sizes between n = 50 and n = 5000, we investigate the inclusion and exclusion of geofactors and the diversity of the resulting models as a function of sample size; the multiplicity of different models is assessed using numerical indices borrowed from information theory and biodiversity research. Model diversity decreases with increasing sample size and reaches either a local minimum or a plateau; even larger sample sizes do not further reduce it, and approach the upper limit of sample size given, in this study, by the autocorrelation range of the spatial datasets. In this way, an optimised sample size can be derived from an exploratory analysis. Model uncertainty due to sampling and model selection, and its predictive ability, are explored statistically and spatially through the example of 100 models estimated in one study area and validated in a neighbouring area: depending on the study area and on sample size, the predicted probabilities for debris flow release differed, on average, by 7 to 23 percentage points. In view of these results, we argue that researchers applying model selection should explore the behaviour of the model selection for different sample sizes, and that consensus models created from a number of random samples should be given preference over models relying on a single sample.

Which data for quantitative landslide susceptibility mapping at operational scale? Case study of the Pays d'Auge plateau hillslopes (Normandy, France)

“…For example, it is not possible to set the ratio between "events" and "no events". Recent research shows the importance of the "no event" sampling in the modelling process (Heckmann et al, 2013), which was not taken into account in this study. In this context the recent developments of coupling GIS and statistical software in a "user friendly" process (Brenning, 2007(Brenning, , 2008 appears as a key factor to transfer the research advances in landslide susceptibility from scientists to stakeholders.…”

Assessing the quality of landslide susceptibility maps – case study Lower Austria

“…The decision to map one point per landslide was aimed at increasing mapping effectiveness, avoiding uncertainty related to mapping landslide polygon boundaries, reducing spatial autocorrelation of the case samples (e.g. landslides) and providing equal treatment of small and large landslide samples (Carrara, 1993;Atkinson and Massari, 1998;Van den Eeckhaut et al, 2006;Heckmann et al, 2013;Petschko et al, 2013a). A comparison of modelling landslide susceptibility with sampling either a single point for the main scarp or a random point anywhere in a landslide polygon was conducted by Petschko et al (2013c).…”

Coğrafi̇ Bi̇lgi̇ Si̇stemi̇ Tabanli Mantiksal Regresyon Yöntemi̇ Kullanilarak Çubuk-Kaleci̇k (Ankara) Şabanözü (Çankiri) Arasinin Heyelan Duyarlilik Değerlendi̇rmesi̇

“…In this case, the logical regression model yields favorable results for the class with high values (Hosmer et al, 2013). Therefore; the modelling is generally made by selecting dependent variables belonging to both classes in equal numbers in logistic regression method (Ayalew and Yamagishi 2005;Duman et al, 2006;Heckmann et al, 2014;Hosmer et al, 2013;Süzen and Doyuran 2004;Yesilnacar and Topal 2005;Nefeslioglu et al, 2008, Tekin 2014, Tekin and Çan 2016. The data set for analysis was formed by combining pixels as much as the pixels that corresponds to the inventory map (180.812 pixels) from the non-sliding regions (3.594.338 pixels) by random selection, so the landslide susceptibility map was produced.…”