Uncertainty and Risk Quantification in High-Dimensional Statistics: Methods for Non-Traditional Settings

Abstract

High-dimensional data are increasingly common across fields such as genomics, economics, and neuroscience, often challenging conventional statistical methods. This dissertation develops new tools for uncertainty quantification in high-dimensional settings where standard assumptions—like sparsity or data homogeneity—may not apply. The first part focuses on high-dimensional causal inference without sparsity. We analyze cross-fitted estimators and derive the asymptotic distribution of the cross-fitted augmented inverse probability weighting (AIPW) estimator under a proportional asymptotics regime. Our results highlight how cross-fitting and regularization impact estimation risk, enabling more accurate inference even in dense, high-dimensional designs. The second part addresses predictive inference under distributional heterogeneity. Classical conformal methods assume identically distributed data, an assumption violated in many real-world applications. We propose conformal algorithms for multi-environment settings, offering valid prediction intervals under minimal assumptions. These methods apply to both regression and classification, support general loss functions, and can incorporate auxiliary information to reduce interval size without compromising coverage. Together, these results advance uncertainty quantification in modern, complex data environments.