Zhang Abstract

"Unveiling mode connectivity of an optimization landscape"

There is typically no "best answer" to the task of topic modeling: contextual interpretations of words and different topic clusterings can lead to very different, and yet satisfactory topic assignments. This is the folk intuition of Variational Inference (VI) performed on Latent Dirichlet Allocation (LDA), but little is known about the  optimization landscape that leads to such results.

Using the string method, we show the existence of maximum energy paths (MEPs) between local optima of the ELBO, the objective function of VI. These paths are essentially flat, implying that local optima are not discrete modes but rather lie on a connected manifold of desirable configurations. This corroborates and extends the empirical experience that topic modeling has many optima, providing an explanation for this "no best answer" phenomenon from the optimization perspective.