Concept Drift Adaptive Physical Event Detection for Social Media Streams

Abstract

Event detection has long been the domain of physical sensors operating in a static dataset assumption. The prevalence of social media and web access has led to the emergence of social, or human sensors who report on events globally. This warrants development of event detectors that can take advantage of the truly dense and high spatial and temporal resolution data provided by more than 3 billion social users. The phenomenon of concept drift, which causes terms and signals associated with a topic to change over time, renders static machine learning ineffective. Towards this end, we present an application for physical event detection on social sensors that improves traditional physical event detection with concept drift adaptation. Our approach continuously updates its machine learning classifiers automatically, without the need for human intervention. It integrates data from heterogeneous sources and is designed to handle weak-signal events (landslides, wildfires) with around ten posts per event in addition to large-signal events (hurricanes, earthquakes) with hundreds of thousands of posts per event. We demonstrate a landslide detector on our application that detects almost 350% more landslides compared to static approaches. Our application has high performance: using classifiers trained in 2014, achieving event detection accuracy of 0.988, compared to 0.762 for static approaches.

Temporal Generalizability in Multimodal Misinformation Detection

“…see Kouw and Loog (2018)). Works such as Suprem et al (2019) and Žliobaitė (2010) have used continuous/incremental learning to train models to respond to "concept drift". In the context of misinformation detection, what is defined as "in-domain" and "out-of-domain" can vary.…”

Evaluating Generalizability of Fine-Tuned Models for Fake News Detection

“…Concept drift occurs when testing or prediction data exhibits distribution shift [14], either in the data domain, or in the label domain [49]. Data domain shift can include introduction of new vocabularies, disappearance of existing words, and word polysemy [42]. Label domain shift occurs when the label space itself changes for the same type of data [22,35].…”