Cognitive Personalization in Microtask Design

Paulino, Dennis; Correia, António; Reis, Arsénio; Guimarães, Diogo Lemos; Rudenko, Roman; Nunes, Carlos; Silva, T. M. S.; Barroso, João; Paredes, Hugo

doi:10.1007/978-3-031-05028-2_5

Cited by 2 publications

(3 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two ways to achieve cognitive personalization in microtask design: administering cognitive tests to assess capabilities, or using task fingerprinting. The former involves transforming psychometric tests into microtasks, a method successfully used in previous studies [ 32 , 33 , 38 , 39 ]. The latter involves evaluating the behavior of crowd workers while performing microtasks, which can be done by analyzing click details or keyboard press data [ 26 , 44 ].…”

Section: Methodsmentioning

confidence: 99%

“…For this purpose, two interfaces were designed: one without any personalization and implying a normal difficulty, and the other one with personalization, where the content or the input elements were adapted to make it easier to solve each microtask. The elements required to make user interface (UI) adaptations were based on the ontology for cognitive personalization in a crowd work context proposed by Paulino and colleagues [ 38 ]. The cognitive knowledge that is represented in the ontology is based on mental functions, as defined by the international classification of functioning, disability and health (ICF), a framework for the classification of health and disability [ 1 ].…”

Section: Methodsmentioning

confidence: 99%

“…Moreover, Sampath et al [ 37 ] found that performance in text-transcription tasks can be improved significantly if the microtask design takes into account working memory or visual saliency. Paulino and colleagues [ 38 , 39 ] suggested that cognitive styles could be used to infer information processing preferences in a crowd work setting. These preferences can then be used to personalize the microtask interface.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Model for Cognitive Personalization of Microtask Design

Paulino

Guimarães

Correia

et al. 2023

Sensors

View full text Add to dashboard Cite

The study of data quality in crowdsourcing campaigns is currently a prominent research topic, given the diverse range of participants involved. A potential solution to enhancing data quality processes in crowdsourcing is cognitive personalization, which involves appropriately adapting or assigning tasks based on a crowd worker’s cognitive profile. There are two common methods for assessing a crowd worker’s cognitive profile: administering online cognitive tests, and inferring behavior from task fingerprinting based on user interaction log events. This article presents the findings of a study that investigated the complementarity of both approaches in a microtask scenario, focusing on personalizing task design. The study involved 134 unique crowd workers recruited from a crowdsourcing marketplace. The main objective was to examine how the administration of cognitive ability tests can be used to allocate crowd workers to microtasks with varying levels of difficulty, including the development of a deep learning model. Another goal was to investigate if task fingerprinting can be used to allocate crowd workers to different microtasks in a personalized manner. The results indicated that both objectives were accomplished, validating the usage of cognitive tests and task fingerprinting as effective mechanisms for microtask personalization, including the development of a deep learning model with 95% accuracy in predicting the accuracy of the microtasks. While we achieved an accuracy of 95%, it is important to note that the small dataset size may have limited the model’s performance.

show abstract