Learning Human Proxy Functions to Optimize Machine Learning Systems for Sociotechnical Context

Abstract

Machine learning (ML) systems are increasingly becoming integrated components of human decision making, but many models are developed without considering the sociotechnical context in which they will operate–i.e. how will they be used by humans to solve specific tasks. This mis-match between the development procedure that does not explicitly consider human context, and these models’ eventual use by humans, often causes their performance to fall short of expectations (e.g.21,48). However optimizing a system to perform well in a specific sociotechnical context requires formalizing the human part of that context in ways that permit optimization. One common approach to this problem is to mathematically specific a proxy function for the desired human property that can be easily optimized and evaluated at scale. An intermediary approach between requiring the user to specify their human property at each point where it is needed, and approximating this property by hand-specifying a proxy function is to learn a proxy function based on querying a user to evaluate the implicit human function defined by the property at a small number of key points. In this thesis, we formalize an optimization framework that describes this category of approaches; lay out human and machine related desiderata to consider when formalizing a particular instance of a problem under this framework; and present and discuss 3 methods that instantiate this framework to solve different problems in human-ML collaboration.