A Lighting-Invariant Approach to Local Shape from Shading

Abstract

Shape from shading is a classical problem in computer vision, in which the depth field of an object or a scene is reconstructed from a pattern of intensities in an image. This can be thought of in some sense as the inverse problem to geometry-based graphics rendering. In this context, shading is defined as a function of illumination, surface geometry, and surface characteristics like pattern or texture. Despite its enduring presence in the field, shape from shading is still largely unresolved. This dissertation shows that under the conventional diffuse shading model with unknown directional lighting, the set of quadratic surface shapes that are consistent with the spatial derivatives of intensity at a single image point is a two-dimensional algebraic variety embedded in the five-dimensional space of quadratic shapes. This work rigorously defines a family of such varieties, describes its geometry, and algebraically proves existence and uniqueness results in the areas of two-shot uncalibrated photometric stereo and coquadratic shape from shading. This work introduces a concise, feedforward model that computes an explicit, differentiable approximation of the variety from the intensity and its derivatives at any single image point. The result is a parallelizable processor that operates at each image point and produces a lighting-invariant descriptor of the continuous set of compatible surface shapes at the point. This processor is demonstrated on the two aforementioned application areas.