Publications -> Conference Papers

Efficient Crowd-Powered Active Learning for Reliable Review Evaluation


Authors: X. Min, Y. Shi, L. Cui, H. Yu, and C. Miao
Title: Efficient Crowd-Powered Active Learning for Reliable Review Evaluation
Abstract: To mitigate uncertainty in the quality of online purchases (e.g., e-commerce), many people rely on review comments from others in their decision-making processes. The key challenge in this situation is how to identify useful comments among a large corpus of candidate review comments with potentially varying usefulness. In this paper, we propose the Reliable Review Evaluation Framework (RREF) which combines crowdsourcing with machine learning to address this problem. To improve crowdsourcing quality control, we propose a novel review query crowdsourcing approach which jointly considers workers’ track records in review provision and current workloads when allocating review comments for workers to rate. Using the ratings crowdsourced from workers, RREF then enhances the adaptive topic classification model selection and weighting functions of AdaBoost with dynamic keyword list reconstruction. RREF has been compared with state-of-the-art related frameworks using a large-scale real-world dataset, and demonstrated over 50%reduction in average classification errors.
Keywords: Crowdsourcing; Review evaluation; Machine learning
Conference Name: 2nd International Conference on Crowd Science and Engineering (ICCSE'17)
Location: Beijing, China
Publisher: ACM
Year: 2017
Accepted PDF File: Efficient_Crowd-Powered_Active_Learning_for_Reliable_Review_Evaluation_accepted.pdf
Permanent Link: https://doi.org/10.1145/3126973.3129307
Reference: X. Min, Y. Shi, L. Cui, H. Yu, and C. Miao, “Efficient crowd-powered active learning for reliable review evaluation,” in Proceedings of the 2nd International Conference on Crowd Science and Engineering (ICCSE’17). ACM, July 2017, pp. 136–143.
bibtex: 
@inproceedings{LILY-c125, 
    author = {Min, Xinping and Shi, Yuliang and Cui, Lizhen and Yu, Han and Miao, Chunyan},
    title  = {Efficient Crowd-Powered Active Learning for Reliable Review Evaluation},  
    booktitle = {Proceedings of the 2nd International Conference on Crowd Science and Engineering (ICCSE'17)}, 
    year  = {2017}, 
    month = {July}, 
    pages = {136-143}, 
    location = {Beijing, China},
    publisher = {ACM},
 }