Reading tea leaves: How humans interpret topic models

TitleReading tea leaves: How humans interpret topic models
Publication TypeConference Papers
Year of Publication2009
AuthorsChang J, Boyd-Graber J, Gerrish S, Wang C, Blei DM
Conference NameProceedings of the 23rd Annual Conference on Neural Information Processing Systems
Date Published2009///

Probabilistic topic models are a popular tool for the unsupervised analysis of text, providing both a predictive model of future text and a latent topic representation of the corpus. Practitioners typically assume that the latent space is semantically meaningful. It is used to check models, summarize the corpus, and guide explo- ration of its contents. However, whether the latent space is interpretable is in need of quantitative evaluation. In this paper, we present new quantitative methods for measuring semantic meaning in inferred topics. We back these measures with large-scale user studies, showing that they capture aspects of the model that are undetected by previous measures of model quality based on held-out likelihood. Surprisingly, topic models which perform better on held-out likelihood may infer less semantically meaningful topics.