Accurate Attribute Mapping from Volunteered Geographic Information: Issues of Volunteer Quantity and Quality

Abstract

Crowdsourcing is a popular means of acquiring data, but the use of such data is limited by concerns with its quality. This is evident within cartography and geographical sciences more generally, with the quality of volunteered geographic information (VGI) recognized as a major challenge to address if the full potential of citizen sensing in mapping applications is to be realized. Here, a means to characterize the quality of volunteers, based only on the data they contribute, was used to explore issues connected with the quantity and quality of volunteers for attribute mapping. The focus was on data in the form of annotations or class labels provided by volunteers who visually interpreted an attribute, land cover, from a series of satellite sensor images. A latent class model was found to be able to provide accurate characterizations of the quality of volunteers in terms of the accuracy of their labelling, irrespective of the number of cases that they labelled. The accuracy with which a volunteer could be characterized tended to increase with the number of volunteers contributing but was typically good at all but small numbers of volunteers. Moreover, the ability to characterize volunteers in terms of the quality of their labelling could be used constructively. For example, volunteers could be ranked in terms of quality which could then be used to select a sub-set as input to a subsequent mapping task. This was particularly important as an identified subset of volunteers could undertake a task more accurately than when part of a larger group of volunteers. The results highlight that both the quantity and quality of volunteers need consideration and that the use of VGI may be enhanced through information on the quality of the volunteers derived entirely from the data provided without any additional information.

A review of volunteered geographic information quality assessment methods

“…The results are used as a reference of quality for contributor and photo features of Flickr, and thereby used to derive credibility indicators. Foody et al (2014) use Geowiki as the data source, where it contains a series of satellite imagery. Volunteered contributors were given the task to label the land use categories in these satellite imagery from a pre-defined set of labels.…”

Crowdsourcing, Citizen Science or Volunteered Geographic Information? The Current State of Crowdsourced Geographic Information

“…A diverse range of approaches exist for quality assessment and control, e.g., [69][70][71], and guidelines for some applications are emerging [72].…”

Relasphone—Mobile and Participative In Situ Forest Biomass Measurements Supporting Satellite Image Mapping

“…Several studies point out that not only quantity [68], but quality of volunteered geographical information (VGI) is crucial [69][70][71]. The overall quality of Relasphone measurements was good in both sites, but can be improved for birch in Finland and Quercus and other non-dominant tree species in Mexico.…”