Efficient Passive Ranging with Computational Optics

Abstract

Passive ranging is using the measurement of environmental light to estimate the distance of the nearest reflective surface from a viewpoint along a direction without emitting photons. It outputs a two-dimensional depth map that represents the distance profile of the scene to the viewpoint.

Nature contains diverse solutions to efficient, passive ranging. Invertebrates, such as jumping spiders and praying mantises, often have specialized optics and neural wirings synergized to perceive range information from the environment. Given their size and power consumption, these creatures demonstrate perfect examples of what scientists would like to achieve in artificial systems -- depth sensors requiring little power, small size, and short latency.

This research invents visual sensors with co-designed optics and computation for efficient, passive ranging. The specially-engineered optics encode depth information into captured images in a way that can be decoded with very few multiplies-and-adds from the processor. As the encoding process is a part of the computation, optics in such platforms are named computational optics.

The thesis presents three monocular depth sensors in this document. They use computational optics ranging from liquid membrane lens to nanophotonics. By partially distributing the computation to optics, these passive sensors require little computational power. The most efficient one predict depth and its confidence with lower than 700 floating point operations per pixel. The sensors have pros and cons in different aspects, such as whether requiring mechanical actuation, whether predicting per-pixel 3D velocity or confidence information along with depth, whether estimating depth from multiple shots or single shot of the environmental light, etc. The prototypes demonstrate a family of complementary solutions to efficient, passive ranging that can be used in various applications.

This research not only provides potential solutions to efficient passive ranging, but also establishes a new paradigm of designing visual sensors by combining computational optics and vision algorithms.