Temporal context-aware task recommendation in crowdsourcing systems

Abstract

In crowdsourcing systems, tasks are distributed to networked people for completion. To ensure the output quality, current crowdsourcing systems highly rely on redundancy of answers provided by multiple workers, however massive redundancy is very expensive. Task recommendation can help requesters to receive good quality output quicker as well as help workers to find their right tasks faster. In our previous works, we proposed a task recommendation framework which performs a factor analysis based on probabilistic matrix factorization (PMF) with which the worker and task latent feature spaces are learned. Our framework adopts active learning on our factor analysis model to minimize the number of task assignments to achieve a target output quality. Moreover, our framework adopts an online-updating approach on model update process to greatly improve the system performance in terms of the running time of model update and the prediction accuracy. However, all previous works on task recommendation in crowdsourcing systems do not consider the temporal change of workers’ preference on tasks, thus cannot make recommendations depending on fresh and novel workers’ preference on tasks. In this paper, we propose a time-aware task recommendation framework in crowdsourcing systems, called Time-Aware TAsk RECommendation (TaTaRec). Our factor analysis model considers both worker task selection preference and worker performance history with a special constraint on the time dimension where the weighting of worker task selection preference gradually decreases over time. Complexity analysis shows that our model is efficient and is scalable to large datasets. We carry out comprehensive experiments on our framework by using both real-world datasets and synthetic datasets to evaluate the performance of our framework and the effects of various parameters on the behaviors of our framework. To the best of our knowledge, we are the first one to propose a task recommendation framework that considers the time aspect of workers’ preference on tasks.