Publications -> Conference Papers

An Incentive Mechanism to Elicit Truthful Opinions for Crowdsourced Multiple Choice Consensus Tasks


Authors: S. Liu, C. Miao, Y. Liu, H. Yu, J. Zhang, and C. Leung
Title: An Incentive Mechanism to Elicit Truthful Opinions for Crowdsourced Multiple Choice Consensus Tasks
Abstract: Crowdsourcing is a rapidly growing technology to harness human intelligence to solve problems that are not suitable for automation. It is especially suitable for consensus tasks which collect opinions from human workers to gain insight into real-world phenomena. In these tasks, motivating workers to provide their truthful opinions is a challenging problem. Most existing incentive mechanisms proposed to address this problem assume that workers share a common prior. However, this assumption is not always valid in practice, resulting in that the existing approaches may discourage workers to provide their truthful opinions. In this paper, we propose a novel incentive mechanism – iMET – to elicit truthful opinions for crowdsourced multiple-choice consensus tasks. By incorporating the concepts of worker credibility and similarity, iMET rewards workers for providing truthful opinions without assuming a common prior. Through extensive simulations on the basis of a collected real-world dataset, iMET has been demonstrated to outperform another two widely used incentive mechanisms in eliciting truthful opinions, especially when diverse truthful opinions are held by workers.
Keywords: Incentive mechanism; Truthful; Crowdsourcing; Consensus tasks
Conference Name: 2015 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT'15)
Location: Singapore, Singapore
Publisher: IEEE
Year: 2015
Accepted PDF File: An_Incentive_Mechanism_to_Elicit_Truthful_Opinions_for_Crowdsourced_Multiple_Choice_Consensus_Tasks_accepted.pdf
Permanent Link: http://dx.doi.org/10.1109/WI-IAT.2015.46
Reference: S. Liu, C. Miao, Y. Liu, H. Yu, J. Zhang, and C. Leung, “An incentive mechanism to elicit truthful opinions for crowdsourced multiple choice consensus tasks,” in Proceedings of the 2015 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT’15). IEEE, December 2015, accepted for publication.
bibtex: 
@inproceedings{LILY-c56, 
    author	= {Liu, Siyuan and Miao, Chunyan and Liu, Yuan and Yu, Han and Zhang, Jie and Leung, Cyril},
    title	= {An Incentive Mechanism to Elicit Truthful Opinions for Crowdsourced Multiple Choice Consensus Tasks},  
    booktitle	= {Proceedings of the 2015 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT'15)}, 
    year		= {2015}, 
    month	= {December}, 
    pages	= {96-103}, 
    location	= {Singapore, Singapore},
    publisher	= {IEEE},
 }