Accelerating Inference: Mitotic Stein Variational Gradient Descent for Bayesian Analysis of Dynamical Systems

Abstract

This thesis introduces mitotic Stein variational gradient descent (mSVGD), a novel enhancement to Stein variational gradient descent (SVGD) designed to improve speed, convergence behavior, and robustness of particle-based variational inference. The research focuses on addressing computational inefficiencies in manifold-constrained Gaussian Process (MAGI) inference, a framework for Bayesian inference of ordinary differential equation systems. By leveraging a structured particle expansion approach inspired by mitotic cell division, mSVGD mitigates sensitivity to hyperparameter selection, accelerates convergence, and enhances robustness against noisy and sparse data. Empirical evaluations on the FitzHugh-Nagumo, Hes1 protein, and Lorenz models demonstrate that mSVGD achieves a 30× to 50× speedup over the traditional MAGI implementation that uses Hamiltonian Monte Carlo sampling, while maintaining or improving inference accuracy. The proposed method also exhibits superior consistency in convergence behavior and increased stability compared to SVGD. These results position mSVGD as a scalable and efficient alternative to both traditional MCMC-based inference techniques and standard SVGD. This work contributes to ongoing advancements in variational inference, particularly for dynamical systems with computational constraints, and highlights the potential of mSVGD as a powerful tool for Bayesian inference in complex, real-world scientific applications.