Face Identification in the Internet Era

Abstract

Despite decades of effort in academia and industry, it is not yet possible to build machines that can replicate many seemingly-basic human perceptual abilities. This work focuses on the problem of face identification that most of us effortlessly solve daily. Substantial progress has been made towards the goal of automatically identifying faces under tightly controlled conditions; however, in the domain of unconstrained face images, many challenges remain. We observe that the recent combination of widespread digital photography, inexpensive digital storage and bandwidth, and online social networks has led to the sudden creation of repositories of billions of shared photographs and opened up an important new domain for unconstrained face identification research. Drawing upon the newly-popular phenomenon of “tagging,” we construct some of the first face identification datasets that are intended to model the digital social spheres of online social network members, and we examine various qualitative and quantitative properties of these image sets. The identification datasets we present here include up to 100 individuals, making them comparable to the average size of members’ networks of “friends” on a popular online social network, and each individual is represented by up to 100 face samples that feature significant real-world variation in appearance, expression, and pose. We demonstrate that biologically-inspired visual representations can achieve state-of-the-art face identification performance on our novel frontal and multi-pose face datasets. We also show that the addition of a tree-structured classifier and training set augmentation can enhance accuracy in the multi-pose setting. Finally, we illustrate that the machine-readable “social context” in which shared photos are often embedded can be applied to further boost face identification accuracy. Taken together, our results suggest that accurate automated face identification in vast online shared photo collections is now feasible.