Aggregating Reliable Submissions in Crowdsourcing Systems

Kurup, Ayswarya R; Sajeev, G P; Swaminathan, J

doi:10.1109/access.2021.3127994

Cited by 3 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here l is the number of workers allocated for a task. For estimating the parameter ab j , similar to [5], we use the maximum likelihood approach. We set the threshold for ab j as 0.5.…”

Section: Updating the Parametersmentioning

confidence: 99%

“…Moreover, it has substantial potential to solve the challenges associated with automation processes, which favors their largescale adoption in the future [3,4]. In crowdsourcing, task aggregation has a significant role in deriving true answers from the submissions of workers [5]. In this scheme, the requesters outsource a set of tasks, and the crowdsourcing platform distributes them to workers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Classifying Workers for Mitigating Adversarial Attacks in Crowdsourcing

First,

Second

2023

Preprint

View full text Add to dashboard Cite

Crowdsourcing is adopted as a fast and cost-effective system for human computation and acquiring data for training models in machine learning. Although crowdsourcing has broad applicability, it still has the following challenges. Normally, workers have access to crowdsourcing platforms with simple authentication mechanisms. As a result, malicious workers may get into the system and submit unreliable answers rendering the platform fraudulent. Moreover, when workers perform tasks, their lack of expertise and the difficulty of tasks affects the accuracy. Truth inference benefits the data quality and worker reliability. A truth inference algorithm estimates workers' trustworthiness and correctness of the answers from their responses. Besides, it helps in filtering out low-quality answers. However, due to adversarial attacks by malicious workers, ground truth inference algorithms perform poorly. This research considers how to defend against adversarial attacks on crowdsourcing platforms and improve the truth inference process. The proposed method estimates the trust and reliability scores of the workers and classifies them as normal and malicious workers. Based on this classification , tasks are assigned to the workers. Moreover, these predicted scores are used for inferring the correct answers, thereby improving the ground truth inference. Experiments consistently show that the proposed truth inference method is tolerant to adversarial attacks with competent accuracy.

show abstract

“…Here l is the number of workers allocated for a task. For estimating the parameter ab j , similar to [5], we use the maximum likelihood approach. We set the threshold for ab j as 0.5.…”

Section: Updating the Parametersmentioning

confidence: 99%