System I/O Integrity for Deep Learning

Abstract

Artificial intelligence applications are moving from special purpose, complementary, datacenter-based uses to general, mission-critical, in-the-wild uses. With this change, AI systems will have to operate in and adapt to heterogeneous, sometimes adversarial, environments, and do so over heterogeneous networks. These new applications are harder to control, and require an entire new ecosystem of methods in order to be successful. Chief among these are new AI-specific security and adaptability tools. As applications are deployed on shared infrastructure, they become vulnerable to entirely new forms of attack from adversaries that jeopardize their safe use. As applications are deployed in-the-wild with vastly different environments and subjects, they must be able to adapt and perform well regardless of the environment in which they operate. Finally, just as the applications themselves are becoming highly heterogeneous, the networks over which they must operate are also becoming highly heterogeneous. Inputs to AI systems transmitted over these new types of networks may be routinely and severely corrupted. In this thesis, we introduce the System I/O Integrity framework to make AI systems more trustworthy, adaptable, and robust in uncontrollable deployment environments. System I/O Integrity, or checking that the inference input is consistent with the inference output of the system and assuring that inference input can be handled by system, provides a fundamentally new, general framework to address these problems. The System I/O Integrity mechanisms are themselves enabled by AI, and are low-cost and effective. This makes them well suited to the challenging environments in which they will be applied. We present four applications and techniques demonstrating the System I/O Integrity framework. By checking the I/O integrity of an inference computation executed by an untrusted third party, System I/O Integrity checks can be used as a security mechanism to ensure the third party has not manipulated the inference computation. This is a fundamental new security primitive for the AI age. By checking the integrity between an input from an unknown environment and a number of models each customized to operate in a specific environment, System I/O Integrity checks can be used to select the appropriate model for the unknown environment. This enables a new paradigm to approach machine learning application design: rather than designing a single model to generalize to all environments and settings, numerous environment-specific models can be trained, and the appropriate one can be efficiently selected efficiently at inference time. By assuring that the inference input to AI systems conform to certain requirements, the performance of the AI system can be assured even in extremely noisy environments and over emerging high frequency networks. We introduce a data-driven localization method and a a hyperdimensional-based communication method to demonstrate this. Together, this thesis shows that checking and ensuring system I/O integrity is a powerful general framework to make real world AI deployments more trustworthy, adaptable, and robust in uncontrolled deployment environments.