Publications -> Conference Papers

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge


Authors: Y. Xiong, H. Chen, M. Zhao, and B. An
Title: HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge
Abstract: It has been an open challenge for self-interested agents to make optimal sequential decisions in complex multiagent systems, where agents might achieve higher utility via collaboration. The Microsoft Malmo Collaborative AI Challenge (MCAC), which is designed to encourage research relating to various problems in Collaborative AI, takes the form of a Minecraft mini-game where players might work together to catch a pig or deviate from cooperation, for pursuing high scores to win the challenge. Various characteristics, such as complex interactions among agents, uncertainties, sequential decision making and limited learning trials all make it extremely challenging to find effective strategies. We present HogRider - the champion agent of MCAC in 2017 out of 81 teams from 26 countries. One key innovation of HogRider is a generalized agent type hypothesis framework to identify the behavior model of the other agents, which is demonstrated to be robust to observation uncertainty. On top of that, a second key innovation is a novel Q-learning approach to learn effective policies against each type of the collaborating agents. Various ideas are proposed to adapt traditional Q-learning to handle complexities in the challenge, including state-action abstraction to reduce problem scale, a warm start approach using human reasoning for addressing limited learning trials, and an active greedy strategy to balance exploitation-exploration. Challenge results show that HogRider outperforms all the other teams by a significant edge, in terms of both optimality and stability.
Keywords: 
Conference Name: 32nd AAAI Conference on Artificial Intelligence (AAAI'18)
Location: New Orleans, USA
Publisher: AAAI Press
Year: 2018
Accepted PDF File: HogRider_Champion_Agent_of_Microsoft_Malmo_Collaborative_AI_Challenge_accepted.pdf
Permanent Link: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16385
Reference: Y. Xiong, H. Chen, M. Zhao, and B. An, “HogRider: Champion agent of Microsoft Malmo collaborative AI challenge,” in Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18). AAAI Press, February 2018, pp. 4767–4774.
bibtex: 
@inproceedings{LILY-c143, 
    author	= {Xiong, Yanhai and Chen, Haipeng and Zhao, Mengchen and An, Bo},
    title	= {{HogRider}: Champion Agent of {M}icrosoft {M}almo Collaborative {AI} Challenge},  
    booktitle	= {Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18)}, 
    year		= {2018}, 
    month	= {February}, 
    pages	= {4767-4774}, 
    location	= {New Orleans, USA},
    publisher	= {AAAI Press},
 }