Publications -> Journal Papers

Distributed multi-task classification: a decentralized online learning approach


Authors: C. Zhang, P. Zhao, S. Hao, Y. C. Soh, B. S. Lee, C. Miao, and S. C. H. Hoi
Title: Distributed multi-task classification: a decentralized online learning approach
Abstract: Although dispersing one single task to distributed learning nodes has been intensively studied by the previous research, multi-task learning on distributed networks is still an area that has not been fully exploited, especially under decentralized settings. The challenge lies in the fact that different tasks may have different optimal learning weights while communication through the distributed network forces all tasks to converge to an unique classifier. In this paper, we present a novel algorithm to overcome this challenge and enable learning multiple tasks simultaneously on a decentralized distributed network. Specifically, the learning framework can be separated into two phases: (i) multi-task information is shared within each node on the first phase; (ii) communication between nodes then leads the whole network to converge to a common minimizer. Theoretical analysis indicates that our algorithm achieves a O(sqrt(T)) regret bound when compared with the best classifier in hindsight, which is further validated by experiments on both synthetic and real-world datasets.
Keywords: Decentralized distributed learning; Multi-task learning; Online learning
Journal Name: Machine Learning
Publisher: Springer
Year: 2017
Accepted PDF File: Distributed_multi-task_classification_a_decentralized_online_learning_approach_accepted.pdf
Permanent Link: https://doi.org/10.1007/s10994-017-5676-y
Reference: C. Zhang, P. Zhao, S. Hao, Y. C. Soh, B. S. Lee, C. Miao, and S. C. H. Hoi, “Distributed multi-task classification: a decentralized online learning approach,” Machine Learning, pp. 1–21, November 2017.
bibtex: 
@article {LILY-j53,
    author  = {Zhang, Chi and Zhao, Peilin and Hao, Shuji and Soh, Yeng Chai and Lee, Bu Sung and Miao, Chunyan and Hoi, Steven C. H.},
    title   = {Distributed multi-task classification: a decentralized online learning approach},
    journal  = {Machine Learning},
    year  = {2017},
    month  = {November},
    volume  = {},
    number  = {},
    pages  = {1-21},
    publisher  = {Springer},
 }