Tasks 10/11: Unsupervised Event-based Optical Flow: Depth and Egomotion

Bio: Alex Zhu is a PhD student at the University of Pennsylvania, working with Dr. Kostas Daniilidis. His current research explores novel algorithms for event cameras for computer vision, as well as unsupervised deep learning methods for motion estimation. He received his Bachelors from Duke University, where he worked with Dr. Michael Zavlanos.

Abstract: Event cameras are a novel bio-inspired asynchronous sensing modality that tracks changes in log image intensity. Due to their asynchronous nature, these cameras allow for extremely fast tracking of motion in the image, with temporal resolution up to a megahertz, as well as the ability to work in extremely high dynamic range scenes (140dB vs 60dB for regular cameras). They also have much lower power consumption than traditional cameras. In this talk, we will explore a novel method that allows a convolutional neural network to learn to regress optical flow, as well as egomotion and depth, directly from an event stream in a fully unsupervised manner (without any ground truth labels). Our network learns by using the predicted flow or egomotion and depth to remove the motion blur from the event stream, which we show is a strong supervisory signal, allowing the network to learn accurate motion information. We also show that the predictions generalize extremely well to different scenes, and allow for tracking of very fast objects, as well as in challenging lighting environments.