A global reference database of crowdsourced cropland data collected using the Geo-Wiki platform

Abstract

A global reference data set on cropland was collected through a crowdsourcing campaign using the Geo-Wiki crowdsourcing tool. The campaign lasted three weeks, with over 80 participants from around the world reviewing almost 36,000 sample units, focussing on cropland identification. For quality assessment purposes, two additional data sets are provided. The first is a control set of 1,793 sample locations validated by students trained in satellite image interpretation. This data set was used to assess the quality of the crowd as the campaign progressed. The second data set contains 60 expert validations for additional evaluation of the quality of the contributions. All data sets are split into two parts: the first part shows all areas classified as cropland and the second part shows cropland average per location and user. After further processing, the data presented here might be suitable to validate and compare medium and high resolution cropland maps generated using remote sensing. These could also be used to train classification algorithms for developing new maps of land cover and cropland extent.

Recent Advances in Forest Observation with Visual Interpretation of Very High-Resolution Imagery

“…New campaigns have since been run to collect a global validation data set for cropland (Laso Bayas et al 2017a). At the same time, Geo-Wiki was also set up to run internal data collection campaigns to produce high-quality training and validation data sets for different tasks related to the development of forest maps.…”

“…At this stage, only Collect Earth is fully open, with code available in Github; this is not yet the case for Geo-Wiki and LACO-Wiki although the new mobile version of LACO-Wiki will be open source. Conversely, the data are not necessarily open from Collect Earth as this depends on the group or persons who use the tool for a particular purpose while the data from Geo-Wiki can either be downloaded from the website or can be found in PANGAEA Laso Bayas et al 2017a). The validation data in LACO-Wiki are theoretically open through the licensing agreement that users agree to when registering in the system although individual users can currently keep the data private or share it more broadly.…”

“…Geo-Wiki campaigns have made extensive use of an expert or control data set, i.e., a subset of the sample that is classified by a group of experts; this control data set is then used to calculate the performance of the crowd Fritz et al 2017;Laso Bayas et al 2017a;Lesiv et al 2018a). The information about performance can then be used to weight the data when used in subsequent applications, e.g., giving less weight to those interpreters who performed less well (e.g., Lesiv et al 2018a).…”

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

“…">Current Trends in Training Data (TD) CollectionA large proportion of remote-sensing projects make some use of TD, typically created either using geolocated in situ data [46,60], by visually interpreting high and/or very high spatial-resolution imagery [26,61,62], or by interpreting the images to be classified/modeled themselves, e.g., [55,56,63,64]. Of these collection methods, HR/VHR image interpretation is increasingly common [65], particularly with the rise in crowdsourcing initiatives [22,66]. As such, mapping is strongly constrained by the creation of TD, which, much like map reference data, are often treated as absolute "truth", in that their accuracy is assumed to be perfect [30,38,47,67].…”

“…Additionally, the increasingly popular large-scale, high-complexity NNs require substantially more TD than traditional statistical models, and like many ML approaches are sensitive to noisy and biased data, producing the logistical difficulty of creating very large, "clean" training datasets [69][70][71].Partially to address this need, several recent efforts have been devoted to producing extremely large training datasets that can be used across a wide range of mapping applications, and to serve as comprehensive benchmarks [72,73]. Similarly, a recent trend has emerged in large-scale mapping projects to employ large teams of TD interpreters, often within citizen science campaigns that rely on web-based data creation tools [22,[74][75][76].…”