Publications -> Conference Papers

Bringing Reputation-awareness into Crowdsourcing


Authors: H. Yu, Z. Shen, and C. Leung
Title: Bringing Reputation-awareness into Crowdsourcing
Abstract: In crowdsourcing systems, worker selection is a challenging issue. Researchers have advocated the incorporation of reputation management into crowdsourcing systems to address this issue. However, current reputation-based decisions often result in concentrating the delegation of human intelligence tasks (HITs) to a small number of highly reputable workers to reduce risk. It conflicts with the main objective of crowdsourcing systems which is to promote mass collaboration. In this paper, we propose a situation-aware approach - SWORD - to enable existing reputation models to work in crowdsourcing systems. We holistically consider the objectives of all stake-holders in a crowdsourcing system (including requesters, workers, and system operators) to formulate the HIT allocation problem as a trade-off between quality and timeliness, and propose an efficient approach via constraint optimization to produce solutions to this problem with low computational complexity. Extensive simulations designed based on the actual conditions from Amazon's Mechanical Turk system demonstrates significant advantage of SWORD compared to existing approaches in improving social welfare.
Keywords: Computational complexity; Social networking (online); Ubiquitous computing
Conference Name: 9th International Conference on Information, Communications and Signal Processing (ICICS'13)
Location: Tainan, Taiwan
Publisher: IEEE
Year: 2013
Accepted PDF File: Bringing_Reputation-awareness_into_Crowdsourcing_accepted.pdf
Permanent Link: http://dx.doi.org/10.1109/ICICS.2013.6782912
Reference: H. Yu, Z. Shen, and C. Leung, “Bringing reputation-awareness into crowdsourcing,” in Proceedings of the 9th International Conference on Information, Communications and Signal Processing (ICICS’13). IEEE, December 2013, pp. 1–5.
bibtex: 
@inproceedings{LILY-c14, 
   author	= {Yu, Han and Shen, Zhiqi and Leung, Cyril}, 
   title	= {Bringing Reputation-awareness into Crowdsourcing}, 
   booktitle	= {Proceedings of the 9th International Conference on Information, Communications and Signal Processing (ICICS'13)}, 
   year		= {2013}, 
   month	= {December}, 
   pages	= {1-5}, 
   location 	= {Tainan, Taiwan},
   publisher	= {IEEE},
}