Selectively Damped Oscillatory State-Space Models: Enabling Adaptive Memory in Sequence Modeling

Abstract

Advances in sequence modeling have emphasized the importance of efficiently capturing long-range dependencies while maintaining stability and computational efficiency. Structured state space models (SSMs) have emerged as a promising framework in this setting, and recent work has shown that oscillatory variants such as LinOSS and D-LinOSS provide a stable and expressive foundation for sequence modeling through second-order dynamics. However, these models rely on fixed damping mechanisms, limiting their ability to adapt memory behavior across time. In this work, we introduce selectively damped LinOSS (SD-LinOSS), an extension of oscillatory state space models that incorporates input-dependent damping. This modification yields a time-varying dynamical system that can adaptively control the balance between memory retention and forgetting based on the input. We show that this formulation preserves the stability and efficiency of prior models while expanding their expressive capabilities, enabling the system to realize trajectories of eigenvalues within the unit disk rather than a fixed spectrum. We provide a theoretical analysis of the resulting model, including characterization of its spectral properties, sufficient conditions for global stability, and an expressivity argument demonstrating that SD-LinOSS strictly generalizes both LinOSS and D-LinOSS. Empirically, we evaluate the model on synthetic and real-world sequence modeling tasks. Our results show that adaptive damping leads to consistent improvements in settings requiring selective memory and long-range dependency modeling, while remaining competitive in regimes where simpler dynamics are sufficient.