Auxiliary-task Based Deep Reinforcement Learning for Participant Selection Problem in Mobile Crowdsourcing

Abstract

In mobile crowdsourcing (MCS), the platform selects participants to complete location-aware tasks from the recruiters aiming to achieve multiple goals (e.g., profit maximization, energy efficiency, and fairness). However, different MCS systems have different goals and there are possibly conflicting goals even in one MCS system. Therefore, it is crucial to design a participant selection algorithm that applies to different MCS systems to achieve multiple goals. To deal with this issue, we formulate the participant selection problem as a reinforcement learning problem and propose to solve it with a novel method, which we call auxiliary-task based deep reinforcement learning (ADRL). We use transformers to extract representations from the context of the MCS system and a pointer network to deal with the combinatorial optimization problem. To improve the sample efficiency, we adopt an auxiliary-task training process that trains the network to predict the imminent tasks from the recruiters, which facilitates the embedding learning of the deep learning model. Additionally, we release a simulated environment on a specific MCS task, the ride-sharing task, and conduct extensive performance evaluations in this environment. The experimental results demonstrate that ADRL outperforms and improves sample efficiency over other well-recognized baselines in various settings. CCS CONCEPTS • Information systems → Crowdsourcing; • Theory of computation → Sequential decision making.

Deep reinforcement learning using least‐squares truncated temporal‐difference

“…Contrastive learning [29] is another way to accelerate feature representation learning with auxiliary self-supervised learning tasks [19]. Some other algorithms utilise auxiliary tasks of explicit predictive models, such as predicting the forward model [16][17][18], predicting the backward model [30], predicting the reward function [31] and introducing other prior knowledge about the model [32,33]. In order to extract task-relevant low-dimensional features from original images.…”

Deep Learning for Mobile Crowdsourcing Techniques, Methods, and Challenges: A Survey

“…In view of the different goals in different mobile crowdsourcing systems, Shen et al [26] designed a participant selection algorithm, which was applied to different mobile crowdsourcing systems to achieve multiple goals and formulate the participant selection problem as a reinforcement learning problem.…”

Transfer Learning in Conversational Analysis through Reusing Preprocessing Data as Supervisors

“…(3 and 4) Using features at different historical timepoints is a common practice in statistical learning, especially in time-series modelling (Christ et al, 2018). Lastly, predicting future labels as auxiliary tasks can help in learning (Caruana et al, 1996;Cooper et al, 2005;Trinh et al, 2018;Zhu et al, 2020;Shen et al, 2020). We propose using historical and future (up to four talkturns ago or later) target labels as auxiliary targets.…”