The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping

Abstract

Abstract:In the past few years, volunteers have produced geographic information of different kinds, using a variety of different crowdsourcing platforms, within a broad range of contexts. However, there is still a lack of clarity about the specific types of tasks that volunteers can perform for deriving geographic information from remotely sensed imagery, and how the quality of the produced information can be assessed for particular task types. To fill this gap, we analyse the existing literature and propose a typology of tasks in geographic information crowdsourcing, which distinguishes between classification, digitisation and conflation tasks. We then present a case study related to the "Missing Maps" project aimed at crowdsourced classification to support humanitarian aid. We use our typology to distinguish between the different types of crowdsourced tasks in the project and choose classification tasks related to identifying roads and settlements for an evaluation of the crowdsourced classification. This evaluation shows that the volunteers achieved a satisfactory overall performance (accuracy: 89%; sensitivity: 73%; and precision: 89%). We also analyse different factors that could influence the performance, concluding that volunteers were more likely to incorrectly classify tasks with small objects. Furthermore, agreement among volunteers was shown to be a very good predictor of the reliability of crowdsourced classification: tasks with the highest agreement level were 41 times more probable to be correctly classified by volunteers. The results thus show that the crowdsourced classification of remotely sensed imagery is able to generate geographic information about human settlements with a high level of quality. This study also makes clear the different sophistication levels of tasks that can be performed by volunteers and reveals some factors that may have an impact on their performance.

Volunteered Geographic Information for Disaster Risk Reduction—The Missing Maps Approach and Its Potential within the Red Cross and Red Crescent Movement

“…The crowdsourced mapping tasks of the Missing Maps project are based on deriving geographic information from remotely sensed imagery [9]. Remote global digital volunteers trace base map information, such as roads and buildings.…”

“…For example, mapping events, which are also called "Mapathons", are organised to bring newcomers and experienced contributors together in a social setting. The MapSwipe App [13], which was developed within the Missing Maps project, makes contributing convenient for individual volunteers, who classify remotely sensed imagery into "no building", "building" and "bad image" using their mobile phones [9]. In the next step, detailed geographic information is added through the detailed mapping of buildings in OSM.…”

Geospatial preparedness: Empirical study of the joint effort to provide geospatial support to disaster response

“…MsF considers Missing Maps a clear added value in the field (Gray, Stringer, & Jobanputra, ). Missing Maps has been analyzed in several research works (de Albuquerque, Herfort, & Eckle, ). The disconnection between local communities and mappers brings concerns about the necessary maintenance and updating of the datasets.…”

The Role of Citizen Science in Earth Observation

“…The paper by Albuquerque et al [20] explores the value of crowdsourced data for humanitarian mapping. The authors identify a typology of the tasks used in geographic information crowdsourcing, including classification and digitization (both performed by citizens) and conflation (performed in this article by the research team).…”

“…This will enable near real-time monitoring of changes occurring on the Earth; such data are becoming increasingly attractive since they can be used to build more applications for citizens that will potentially stimulate their engagement and further mainstream the use of satellite data by citizens in their daily lives. [20] Disaster Online ** -** --* F, C Busch el al. [21] Marine Field-based ** ** * + + * F, P, C Elmore et al [28] Phenology Field-based *** ** + -* C Heigl et al [22] Biodiversity Field-based *** *** ** C Kosmala el al.…”