Task 019: Robustness of Neural Systems

Abstract: 

Spiking Neural Networks (SNNs) have gained huge attention as a potential energy-efficient alternative to conventional Artificial Neural Networks (ANNs) due to their inherent high-sparsity activation. However, most prior SNN methods use forward-only ANN-like architectures (e.g., VGG-Net or ResNet), which could provide sub-optimal performance for temporal sequence processing of binary information in SNNs. To address this, we introduce a novel Neural Architecture Search (NAS) approach for finding better SNN architectures. Inspired by recent NAS approaches that find the optimal architecture from activation patterns at initialization, we select the architecture that can represent diverse spike activation patterns across different data samples without training. Moreover, to further leverage the temporal information among the spikes, we search for feed-forward as well as backward connections (i.e., temporal feedback connections) between layers. Through extensive experiments on CIFAR10, CIFAR100 and Tiny ImageNet, the backward connection architectures founded by our search algorithm achieve SOTA performance, demonstrating the importance of designing SNN architecture using temporal information properly. Our code can be found at https://github.com/Intelligent-Computing-Lab-Yale/Neural-Architecture-Search-for-Spiking-Neural-Networks.

 

Bio: Youngeun Kim is currently working toward a Ph.D. degree in Electrical Engineering at Yale University,  advised by Prof. Priyadarshini Panda.  He received his B.S. degree in Electronic Engineering from Sogang University, South Korea, in 2018 and M.S. degree in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST), South Korea, in 2020. His research interests include neuromorphic computing, computer vision, and algorithm-hardware co-design.