Characterizing Posterior Uncertainty for the Indian Buffet Process

Abstract

Many problems in data science and machine learning require identifying latent features that occur in a set of observations. For example, given a set of images, we may want to determine what objects are contained in each image, with the understanding that one image can contain multiple objects and a one object can occur in multiple images. The Indian buffet process (IBP) places a nonparametric prior distribution on the presences of features in a dataset, with no upper bound on the total number of features. Given the complex combinatorial structure of the IBP and the infinite-dimensional space of latent variable values, posterior inference for the IBP is very difficult; most algorithms tends to converge to local optima rather than adequately characterizing posterior uncertainty. This thesis aims to develop better methods for analyzing highly uncertain or multimodal posterior distributions in IBP inference. Specifically, we develop a new family of multimodal distributions for variational inference, and derive the variational lower bound. We also discuss the use of the variational bound and the Stein discrepancy for evaluation, and present a heuristic split proposal algorithm for producing multiple factorizations. We conclude by presenting empirical results on synthetic data and discussing avenues for further research. Because the problem of getting trapped in local optima is ubiquitous in nonparametric Bayesian inference, the ideas discussed here would be relevant for inference in a wide variety of models.