Task 005 - Neural Fabrics

ABSTRACT: In-memory computing (IMC) utilizes simultaneous activation of multiple rows and analog computation to greatly improve the energy efficiency of deep neural network (DNNs) hardware. The analog nature of IMC typically induces some amount of drop in the DNN accuracy due to the intrinsic hardware noise, which is considered a negative effect. However, in this work, we discover that such hardware intrinsic noise can, on the contrary, play a positive role in enhancing adversarial robustness. To achieve that, we propose a new DNN training scheme that integrates measured IMC hardware noise and aggressive partial sum quantization at the IMC crossbar. We show that this effectively improves the robustness of IMC DNN hardware against both adversarial input and weight attacks.

 

Training low-precision DNNs with IMC involves obfuscated gradients due to discrete quantization of activations/weights as well as the partial sums, which makes the implementation of white-box adversarial attacks difficult. Hence, we used a black-box substitution method and PGD adversarial attack to generate strong adversarial input examples. On the other hand, we used a gradient-ranking based bit-flip algorithm to perform the adversarial weight attacks. Against black-box adversarial input attacks and bit-flip weight attacks, the robustness of our IMC DNN hardware has improved by up to 10.5% CIFAR-10 accuracy and 33.6% in the number of required bit-flips, respectively, compared to conventional DNNs.

 

Bio: Sai Kiran Cherupally is currently a PhD student at Arizona State University. He joined Prof. Jae-sun Seo’s research group in 2018, and has been involved in multiple research projects such as design of ECG-based hardware security engine, improving deep neural network performance in in-memory computing (IMC) hardware, and leveraging IMC hardware noise to improve the robustness of deep neural networks to adversarial attacks. Sai’s research interests are machine learning assisted hardware security and developing defenses against adversarial attacks using noise-injection based DNN optimization. He is a Student Member of the IEEE.