Living with Interruptions: Resilient and Adaptive Machine Learning Infrastructure

Abstract

As machine learning (ML) systems continue to grow in scale and complexity, the traditional path to performance—scaling up compute—has begun to show diminishing returns. Interruptions have emerged as the dominant performance bottleneck. Software contentions, management events, hardware failures, and network congestion impose recurring overheads that reduce system efficiency and fundamentally threaten the reliability of ML infrastructure. These interruptions manifest as two distinct challenges that current systems have not adequately addressed. At the upper layer, every interruption forces a costly redo—recovery repeatedly triggers expensive initialization that blocks execution progress across the entire job. At the lower layer, shared network fabrics turn transient congestion into persistent throughput degradation. This dissertation argues that resilience and adaptability must be treated as first-class design principles, with solutions targeting each layer directly. To this end, this dissertation presents three systems. TrainMover enables rapid, localized recovery from large-scale training interruptions by decoupling repeated initialization from the execution critical path. Cherry mitigates network contention in shared datacenters by placing packet scheduling and path selection directly in RDMA NIC hardware. DragonEye supports efficient, modeless edge inference under tight resource constraints and lossy network conditions. Together, these systems move beyond static peak optimization, enabling ML infrastructure to operate robustly under persistent runtime instability.