Task 2777.001:Reinforcement learning for efficient maze exploration with sparse rewards

Abstract: While Deep Reinforcement Learning algorithms have shown success in producing autonomous agents that can solve complex tasks, these algorithms have traditionally been premised on the agent receiving dense rewards from the environment. Furthermore, these algorithms are very sample inefficient, requiring a large number of interaction steps with the environment for the agent to effectively learn a desired task. This limits the generalizability of these algorithms to transfer performance to real world scenarios and hardware.

In this work, we present a reinforcement learning approach to solving an exploration task, implemented in the Vizdoom Platform. The agent’s goal is to reach a specific single target, whereupon the agent receives its only reward from the environment.  The agent is trained with an Advantage Actor Critic model, combined with an intrinsic reward signal to overcome the reward sparsity. The intrinsic reward is generated from the predictive error of the agent and thereby encourages exploration to novel states in the environment. By taking advantage of other possible input sources (e.g. depth), we demonstrate how sample efficiency can be improved.  We evaluate this approach on commonly used Vizdoom scenarios, as well as in custom-built environments with additional complexity.

Bio: Juan Carvajal is a PhD student at Purdue University, working with Professor Eugenio Culurciello since Fall 2018. Before joining Purdue, Juan was a research assistant at the Centro de investigación, desarrollo e innovación de sistemas computacionales,  CIDIS, in Ecuador and Fraunhofer IIS, in Germany. He received his Bachelor from University of Navarra, Spain.  His current research is in Efficient Deep Reinforcement Learning.