Publications -> Conference Papers

Generative Topic Embedding: a Continuous Representation of Documents


Authors: S. Li, T.-S. Chua, J. Zhu, and C. Miao
Title: Generative Topic Embedding: a Continuous Representation of Documents
Abstract: Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window. On the other hand, topic modeling maps documents onto a low-dimensional topic space, by utilizing the global word collocation patterns in the same document. These two types of patterns are complementary. In this paper, we propose a generative topic embedding model to combine the two types of patterns. In our model, topics are represented by embedding vectors, and are shared across documents. The probability of each word is influenced by both its local context and its topic. A variational inference method yields the topic embeddings as well as the topic mixing proportions for each document. Jointly they represent the document in a low-dimensional continuous space. In two document classification tasks, our method performs better than eight existing methods, with fewer features. In addition, we illustrate with an example that our method can generate coherent topics even based on only one document.
Keywords: 
Conference Name: 54th Annual Meeting of the Association for Computational Linguistics (ACL’16)
Location: Berlin, Germany
Publisher: Association for Computational Linguistics
Year: 2016
Accepted PDF File: Generative_Topic_Embedding_a_Continuous_Representation_of_Documents_accepted.pdf
Permanent Link: http://aclweb.org/anthology/P/P16/P16-1063.pdf
Reference: S. Li, T.-S. Chua, J. Zhu, and C. Miao, “Generative topic embedding: a continuous representation of documents,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). Association for Computational Linguistics, August 2016, pp. 666–675.
bibtex: 
@inproceedings{LILY-c96, 
    author	= {Li, Shaohua and Chua, Tat-Seng and Zhu, Jun and Miao, Chunyan},
    title	= {Generative Topic Embedding: a Continuous Representation of Documents},  
    booktitle	= {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL'16)}, 
    year		= {2016}, 
    month	= {August}, 
    pages	= {666-675}, 
    location	= {Berlin, Germany},
    publisher	= {Association for Computational Linguistics},
 }