Publications -> Journal Papers

Dynamic and Incremental Exploration Strategy in Fusion Adaptive Resonance Theory for Online Reinforcement Learning


Authors: B. Subagdja
Title: Dynamic and Incremental Exploration Strategy in Fusion Adaptive Resonance Theory for Online Reinforcement Learning
Abstract: One of the fundamental challenges in reinforcement learning is to setup a proper balance between exploration and exploitation to obtain the maximum cummulative reward in the long run. Most protocols for exploration bound the overall values to a convergent level of performance. If new knowledge is inserted or the environment is suddenly changed, the issue becomes more intricate as the exploration must compromise the pre-existing knowledge. This paper presents a type of multi-channel adaptive resonance theory (ART) neural network model called fusion ART which serves as a fuzzy approximator for reinforcement learning with inherent features that can regulate the exploration strategy. This intrinsic regulation is driven by the condition of the knowledge learnt so far by the agent. The model offers a stable but incremental reinforcement learning that can involve prior rules as bootstrap knowledge for guiding the agent to select the right action. Experiments in obstacle avoidance and navigation tasks demonstrate that in the configuration of learning wherein the agent learns from scratch, the inherent exploration model in fusion ART model is comparable to the basic epsilon-greedy policy. On the other hand, the model is demonstrated to deal with prior knowledge and strike a balance between exploration and exploitation.
Keywords: Reinforcement learning; Exploration strategy; Adaptive resonance theory
Journal Name: Journal of Computer Science and Information, vol. 9, no. 2
Publisher: Faculty of Computer Science Universitas Indonesia
Year: 2016
Accepted PDF File: Dynamic_and_Incremental_Exploration_Strategy_in_Fusion_Adaptive_Resonance_Theory_for_Online_Reinforcement_Learning_accepted.pdf
Permanent Link: http://dx.doi.org/10.21609/jiki.v9i2.380
Reference: B. Subagdja, “Dynamic and incremental exploration strategy in fusion adaptive resonance theory for online reinforcement learning,” Journal of Computer Science and Information (JIKI), vol. 9, no. 2, pp. 59–69, June 2016.
bibtex: 
@article {LILY-j26,
   author 	= {Subagdja, Budhitama},
   title 	= {Dynamic and Incremental Exploration Strategy in Fusion Adaptive Resonance Theory for Online Reinforcement Learning},
   journal 	= {Journal of Computer Science and Information (JIKI)},
   year 	= {2016},
   month 	= {June},
   volume 	= {9},
   number 	= {2},
   pages 	= {59-69},
   publisher 	= {Faculty of Computer Science Universitas Indonesia},
}