Group Symmetries in Diffusion Models: Formulation, Generalization, and Enforcement

Abstract

Group symmetries are fundamental structures in many real-world datasets, and lever- aging them is crucial for building robust and data-efficient machine learning models. Diffusion models have achieved state-of-the-art performance in generative tasks but typically rely on standard neural network architectures for score estimation. This thesis investigates whether such standard configurations enable diffusion models to implicitly learn and generalize underlying data symmetries purely from examples, particularly when data is partially observed. Drawing motivation from Neural Tangent Kernel (NTK) theory, which suggests limitations in the ability of standard supervised networks to generalize symmetries beyond local data structure, we hypothesize and empirically demonstrate that score networks in diffusion models exhibit similar constraints. Using a 2D toy dataset with inherent SO(2) rotational symmetry, we show a consistent failure of standard models trained on incomplete data (interpolation and extrapolation settings) to generalize symmetry, exhibiting significant score field distortions in unobserved regions. To address this limitation, we propose and evaluate a novel per-timestep symmetry loss that regularizes the denoising process to encourage approximate equivariance. Empirical results on both the toy dataset and higher-dimensional MNIST data confirm that this loss significantly enhances symmetry generalization even in standard architectures, yielding geometrically consistent results comparable to extensive data augmentation. This work highlights a critical limitation in standard diffusion models and underscores the importance of incorporating explicit geometric biases, via architecture or regularization, for reliable generative modeling on structured data.