The Implementation of Optical Flow in Neural Networks

Abstract

The goal of this project was to explore methods for optical flow and methods for machine learning as separate topics, for the purpose of integrating the two. This integration could then be used in modern motor/robotic systems to help determine object and observer movement (direction and speed) as part of a system of inputs which a computer could then respond to. While there are a few prominent methods for computing optical flow, the main one utilized for this project is known as the Lucas-Kanade method. MATLAB was used to code an implementation of the Lucas-Kanade method, which was then updated over time to include more features and present more accuracy. This code ultimately created a ground truth dataset for over 1000 image pairs from the SINTEL dataset to be used in the training of a neural network. This neural network was coded in Python using Keras, an open source neural network library. The final product of the testing provided results on par with modern image classification neural network benchmarks, and together they produce an easy path for fine-tuning code and adjusting network architecture to produce more desirable, high-accuracy results.