StreamCollab: A Streaming Crowd-AI Collaborative System to Smart Urban Infrastructure Monitoring in Social Sensing

Abstract

Social sensing has emerged as a pervasive and scalable sensing paradigm to collect observations of the physical world from human sensors. A key advantage of social sensing is its infrastructure-free nature. In this paper, we focus on a streaming urban infrastructure monitoring (Streaming UIM) problem in social sensing. The goal is to automatically detect the urban infrastructure damages from the streaming imagery data posted on social media by exploring the collective power of both AI and human intelligence from crowdsourcing systems. Our work is motivated by the limitation of current AI and crowdsourcing solutions that either fail in many critical time-sensitive UIM application scenarios or are not easily generalizable to monitor the damage of different types of urban infrastructures. We identify two critical challenges in solving our problem: i) it is difficult to dynamically integrate AI and crowd intelligence to effectively identify and fix the failure cases of AI solutions; ii) it is non-trivial to obtain accurate human intelligence from unreliable crowd workers in streaming UIM applications. In this paper, we propose StreamCollab, a streaming crowd-AI collaborative system that explores the collaborative intelligence from AI and crowd to solve the streaming UIM problem. The evaluation results on a real-world urban infrastructure imagery dataset collected from social media demonstrate that StreamCollab consistently outperforms both state-of-the-art AI and crowd-AI baselines in UIM accuracy while maintaining the lowest computational cost.

SocialDrought: A Social and News Media Driven Dataset and Analytical Platform towards Understanding Societal Impact of Drought

“…Conventional methods for drought monitoring and impact assessment often leverage satellite imagery, climatic data analysis, and hydrological models to predict drought patterns, assess water resource availability, and understand ecological and socioeconomic impacts (Lindersson et al 2020). More recently, social media and news media have been increasingly utilized as informational sources to obtain timely observations from massive social media users and professional journalists in estimating the impact of drought, especially for its impact on human society (Zhang et al 2021a;Shang et al 2022a). For example, MARIC (2022) developed a text classification model to categorize drought-related tweets into predefined categories to study the trends of drought impact.…”

A Crowd-AI Collaborative Duo Relational Graph Learning Framework towards Social Impact Aware Photo Classification

“…Our tasks are deployed on Amazon MTurk, which randomly selects online crowd workers to answer the tasks regardless of their demographic attributes (e.g., race, gender, age) (Zhang et al 2021a). We only allow crowd workers who have a 95% or higher Human Intelligence Task (HIT) approval rate to answer our tasks to ensure the quality of the answers.…”