Psychometrically-informed computer vision models of human face processing

Abstract

We develop a new approach to computer vision using the techniques of psychophysics. Our system provides insight into the role of visual features in face perception, as well as the role some aspects of face perception — judgments of personality — play in everyday life. In the introduction we situate the philosophy of our method historically. In Chapter One, we describe our method and apply it to detecting faces in images. We achieve state-of-the-art performance on a very difficult face detection benchmark. In Chapter Two, we extend our method to computer vision models that mimic human judgments of personality traits from face images. Our models are able to capture much of the variance in human performance. We show evidence for the hypothesis that human judgments of these traits can be driven by low-level image characteristics of the particular photographs used, as opposed to invariant characteristics of face physiognomy. This presents a complication to the standard explanation of judgments of personality from face images. In Chapter Three, we build on the literature that shows correlations between impressions of personality and outcomes. We attempt to use our models of personality to predict success for YouTube videos. This approach, though a priori plausible, explains very little if any of the variance in view rates. We discuss why this prediction failed. Concluding, we discuss future directions where practical uses for our technique have been shown.