Performance of Players and Data Quality in a Citizen Science Casual Game

Abstract

Background: Citizen science games are a type of Games with a Purpose (GWAPs), whose aim is to harness the skills of volunteers for solving scientific problems or contributing to action projects, where citizens intervene in social concerns. Employing games to collect data, classify images or even solve major scientific problems is a relatively new but growing phenomenon in citizen science. A main concern in citizen science is to ensure data quality. As games can be seen as having adverse effects on data quality, it is important to understand how citizen scientists produce data using games, how accurate this data can be, and whether and how games influence data quality. Objective: The objective of this study was to evaluate the performance of individual players’ data quality in MalariaSpot, a citizen science casual game in which volunteers are tasked with detecting parasites in digitized blood sample images.Methods: We used descriptive statistics to analyze a subset of the gameplays recorded and stored in the MalariaSpot database, comparing its clicks to the Gold Standard position of the parasites. This subset includes 15,546 gameplays played over 38 known images that correspond to 97,200 clicks from 1,278 different players. Gameplays have been played via the Android and iOS applications and via the web version of the game. Images were acquired in three different locations and therefore sample preparation have been done by different lab technicians. Two distinct technologies were used for sample digitalization.Results: The overall values for sensibility, specificity, and accuracy of the individual gameplays for the 38 images are 0.82, 0.60, and 0.29 respectively. High presence of parasites in an image makes it easier for players to detect them as their structures tend to look alike and can be compared. Being a simple casual game, the learning curve is very fast and after few minutes, players attend their typical performance level. Data quality is considerably lower in images acquired with mobile phones coupled to the microscope ocular compared to those digitized with standardized digitalization technologies. Conclusions: The results indicate that data quality is influenced by the game, the technologies for image digitalization and the sampling preparation.