Invariance versus Adversarial Learning in Domain Generalization with Applications to Neuroscience

Abstract

We explore the practical application of two modern domain invariant representation-learning techniques for addressing the domain generalization problem in statistical machine learning. Specifically, we investigate the performance of cutting-edge adversarial neural networks and invariance-based invariant risk minimization for producing representations in a real neuroscience data setting. The mappings that are learned at training time are then utilized at test time to produce representations on unlabeled covariates of a held-out distribution shifted domain for a binary classification problem. The results of these experiments are useful in practical contexts including medical diagnosis, object recognition, segmentation, and action recognition as models created/trained on data in these fields suffer from issues of data exchangeability. We typically see this occur when the underlying distributions of collected data tend to vary across geographical location or methodology from study to study. To our knowledge, our results provide first insights into empirical results of purely theoretical modern solutions to this domain generalization problem in the adversarial and invariance learning spaces.