Theme 3: Distributed Intelligence

Abstract:

Estimating an accurate and dense 3D shape of the objects or, environment is essential for scene understanding applications such as autonomous navigation, augmented reality, and robotics. Most of the works to date try to solve this problem using a collection of images or creating volumetric grids which ignore the 3D invariance property of the objects after geometric transformations. Only RGB based techniques suffer from inaccuracies since the ground truth shape is somewhat ambiguous from a single view. Hence, some of the studies use sensor fusion techniques where RGB and dense depth maps are combined through a convolutional neural network to create a 3D point cloud. We argue that this type of solution is not realistic since the assumption of having a dense depth map is not valid in practical case. Traditional LiDAR and Time-of-Flight sensors provide accurate but very sparse data both temporally and spatially. In this work, we propose a 3D shape reconstruction pipeline fusing the camera and the sparse LiDAR data Our proposed architecture not only generates a dense 3D point cloud providing very competitive performance compared to the state-of-the-art 3D reconstruction studies, but also demonstrates a novel way of sensor fusion with the multi-head self-attention blocks.

 

Bio:

Md Fahim Faysal Khan is currently pursuing his Ph.D. degree at the Pennsylvania State University with Prof. Vijaykrishnan Narayanan. His current research focuses on multimodal edge intelligence on resource constrained environments. Throughout his PhD, he has proposed multiple generative models that work with less data and also built frameworks to accelerate such ML workloads through techniques like quantization and pruning. He is passionate about working in the boundary of hardware and software and tackling optimization challenges in this domain. Prior joining to Penn State, he completed his B.Sc. in Electrical and Electronic Engineering (EEE) from Bangladesh University of Engineering and Technology (BUET) in 2017.