Automated digital data acquisition for landslide inventories

Abstract

Landslide research relies on landslide inventories for a multitude of spatial, temporal, or process analyses. Generally, it takes high effort to populate a landslide inventory with relevant data. In this context, the present work investigated an effective way to handle vast amounts of automatically acquired digital data for landslide inventories by the use of machine learning algorithms and information filtering. Between July 2017 and February 2019, a keyword alert system provided 4381 documents that were automatically processed to detect landslide events in Germany. Of all those documents, 91% were automatically recognized as irrelevant or duplicates; thereby, the data volume was significantly reduced to contain only actual landslide documents. Moreover, it was shown that inclusion of the document's images into the automated process chain for information filtering is recommended, since otherwise unobtainable important information was found in them. Compared with manual methods, the automated process chain eliminated personal idiosyncrasies and human error and replaced it with a quantifiable machine error. The applied individual algorithms for natural language processing, information retrieval, and classification have been tried and tested in their respective fields. Furthermore, the proposed method is not restricted to a specific language or region. All languages on which these algorithms are applicable can be used with the proposed method and the training of the process chain can take any geographical restriction into account. Thus, the present work introduced a method with a quantifiable error to automatically classify and filter large amounts of data during automated digital data acquisition for landslide inventories.

Exploring a landslide inventory created by automated web data mining: the case of Italy

“…Systems using automated or real-time updates are still uncommon and only used for some types of natural hazards (Battistini et al 2013, Battistini et al 2017Calvello and Pecoraro 2018), mainly earthquakes, floods and wildfire, while creating a complete and updated database is more difficult for landslide (Galli et al 2008;Santangelo et al 2010). The methodology of Battistini et al (2013Battistini et al ( , 2017 and Kreuzer and Damm (2020) allows to update in near real time the landslide database using the data mining technique inside online newspaper articles.…”

Quantitative assessment of information quality in textual sources for landslide inventories

“…In this context, compilations might be underrepresented and findings for scientific publications are likely to be more robust. The calculated outlier-ratio of 0.51 for (Kreuzer and Damm 2020). This does not affect frequency of occurrence, it rather shows that the recent overall tempo-spatial frequency of "news-worthy" landslides in Germany is usually low enough to be reported on individually by multiple newspaper articles.…”

“…Bibus and Terhorst 2001;Hardenbicker and Grunert 2001;Jäger et al 2018;Klose et al 2014;Schmidt and Beyer 2003;Von der Heyden 2004) and automated digital data acquisition (cf. Kreuzer and Damm 2020). Moreover, information recorded in the dataset that was found in textual sources was manually validated and, if necessary, corrected in case the author of the text was not an expert-this is generally true for all source types except for scientific publications and expert opinion pieces.…”

Optimization of rainfall thresholds for landslide early warning through false alarm reduction and a multi-source validation

“…Data mining is another recently developed technique used to obtain information related to natural hazards from online newspaper articles (Battistini et al 2013(Battistini et al , 2017Kreuzer and Damm 2020). In fact, several studies have verified that mass media are generally the first and primary source of information about hazards for the public (Fischer 1994).…”