Task assignment optimization in knowledge-intensive crowdsourcing

Roy, Senjuti Basu; Lykourentzou, Ioanna; Thirumuruganathan, Saravanan; Amer-Yahia, Sihem; Das, Gautam

doi:10.1007/s00778-015-0385-2

Cited by 103 publications

(27 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same time complexity has also been proven in alternative formulations treating crowdsourcing task delegation as a joint optimization problem, which has resulted in the need for part of the solutions to be computed offline 17 . This makes the solutions unable to keep up with changes in situational factor in real time in crowdsourcing systems.…”

Section: Introductionmentioning

confidence: 99%

Algorithmic Management for Improving Collective Productivity in Crowdsourcing

Miao

Chen

et al. 2017

Sci Rep

View full text Add to dashboard Cite

Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index (WDI) jointly considering the effect of each worker’s reputation, workload and motivation to work on collective productivity. Through evaluating workers’ WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.

show abstract

Section: Introductionmentioning

confidence: 99%

Algorithmic Management for Improving Collective Productivity in Crowdsourcing

Miao

Chen

et al. 2017

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Online crowdsourcing platforms often offer practitioners numerous criteria for selecting workers based on their attributes and experience, which can be used to match workers with specific tasks toward reducing costs by increasing productivity. For example, matching worker expertise and wage requirements with task is shown to enhance knowledge production in collaborative crowdsourcing (Roy et al, 2015). Although many citizen science projects do not collect personal information, it would also be possible to predict individual performance before participants perform tasks by assessing individual attributes through a simple survey.…”

Section: Discussionmentioning

confidence: 99%

“…However, a great effort is required to increase participation (Segal et al, 2015) and data volume (Sprinks et al, 2017), especially when the projects focus on specific topics that may not appeal to broad audiences (Prestopnik & Crowston, 2012). A new approach is in need to leverage the effort of limited pools of participants (Roy et al, 2015) and maximize their potential productivity.…”

Section: Introductionmentioning

confidence: 99%

Matching individual attributes with task types in collaborative citizen science

Nakayama

Torre

Nov

et al. 2019

PeerJ Computer Science

View full text Add to dashboard Cite

In citizen science, participants' productivity is imperative to project success. We investigate the feasibility of a collaborative approach to citizen science, within which productivity is enhanced by capitalizing on the diversity of individual attributes among participants. Specifically, we explore the possibility of enhancing productivity by integrating multiple individual attributes to inform the choice of which task should be assigned to which individual. To that end, we collect data in an online citizen science project composed of two task types: (i) filtering images of interest from an image repository in a limited time, and (ii) allocating tags on the object in the filtered images over unlimited time. The first task is assigned to those who have more experience in playing action video games, and the second task to those who have higher intrinsic motivation to participate. While each attribute has weak predictive power on the task performance, we demonstrate a greater increase in productivity when assigning participants to the task based on a combination of these attributes. We acknowledge that such an increase is modest compared to the case where participants are randomly assigned to the tasks, which could offset the effort of implementing our attribute-based task assignment scheme. This study constitutes a first step toward understanding and capitalizing on individual differences in attributes toward enhancing productivity in collaborative citizen science. How to cite this article Nakayama S, Torre M, Nov O, Porfiri M. 2019. Matching individual attributes with task types in collaborative citizen science. PeerJ Comput. Sci. 5:e209 http://doi.org/10.7717/peerj-cs.209 Nakayama et al. (2019), PeerJ Comput. Sci., DOI 10.7717/peerj-cs.209 2/16 a meta-analysis. Personnel Psychology 44(1):1-26

show abstract

“…FSMs were successfully used to model human-robot interactions and dialogue behaviour [29,30]. Agent-based systems are developed for simulating (virtual) human behaviour in a variety of disciplines, from knowledge building in collaborative online communities, like wikis [31,32] to task assignment in crowd work environments [33][34][35] to the way people select which exhibits to see in the physical space of a museum [36]. Users are represented as intentional rational agents.…”

Section: User Profilingmentioning

confidence: 99%

A Semantic Mixed Reality Framework for Shared Cultural Experiences Ecosystems

Vassilakis

Kotis

Spiliotopoulos

et al. 2020

BDCC

Self Cite

View full text Add to dashboard Cite

This paper presents SemMR, a semantic framework for modelling interactions between human and non-human entities and managing reusable and optimized cultural experiences, towards a shared cultural experience ecosystem that might seamlessly accommodate mixed reality experiences. The SemMR framework synthesizes and integrates interaction data into semantically rich reusable structures and facilitates the interaction between different types of entities in a symbiotic way, within a large, virtual, and fully experiential open world, promoting experience sharing at the user level, as well as data/application interoperability and low-effort implementation at the software engineering level. The proposed semantic framework introduces methods for low-effort implementation and the deployment of open and reusable cultural content, applications, and tools, around the concept of cultural experience as a semantic trajectory or simply, experience as a trajectory (eX-trajectory). The methods facilitate the collection and analysis of data regarding the behaviour of users and their interaction with other users and the environment, towards optimizing eX-trajectories via reconfiguration. The SemMR framework supports the synthesis, enhancement, and recommendation of highly complex reconfigurable eX-trajectories, while using semantically integrated disparate and heterogeneous related data. Overall, this work aims to semantically manage interactions and experiences through the eX-trajectory concept, towards delivering enriched cultural experiences.

show abstract

Task assignment optimization in knowledge-intensive crowdsourcing

Cited by 103 publications

References 39 publications

Algorithmic Management for Improving Collective Productivity in Crowdsourcing

Algorithmic Management for Improving Collective Productivity in Crowdsourcing

Matching individual attributes with task types in collaborative citizen science

A Semantic Mixed Reality Framework for Shared Cultural Experiences Ecosystems

Contact Info

Product

Resources

About