Rotated Mean-Field Variational Inference and Iterative Gaussianization
Sifan Liu (Duke University)
E18-304
Abstract: Mean-field variational inference (MFVI) approximates a target distribution with a product distribution in the standard coordinate system, offering a scalable approach to Bayesian inference but often severely underestimating uncertainty due to neglected dependence. We show that MFVI can be greatly improved when performed along carefully chosen principal component axes rather than the standard coordinates. The principal components are obtained from a cross-covariance matrix of the target’s score function and identify orthogonal directions that capture the dominant discrepancies between the…
