Theme 1: Neuro-inspired Algorithms and Theory
Event Date: | July 6, 2023 |
---|---|
Time: | 11:00 am (ET) / 8:00 am (PT) |
Priority: | No |
College Calendar: | Show |
Aparna Aketi, Purdue University
Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data
Abstract: Decentralized learning enables the training of deep learning models over large distributed datasets generated at different locations, without the need for a central server. However, in practical scenarios, the data distribution across these devices can be significantly different, leading to a degradation in model performance. In this paper, we focus on designing a decentralized learning algorithm that is less susceptible to variations in data distribution across devices. We propose Global Update Tracking (GUT), a novel tracking-based method that mitigates the impact of heterogeneous data in decentralized learning without introducing any communication overhead. We theoretically show that the convergence of the proposed algorithm matches with the state-of-the-art decentralized methods. We demonstrate the effectiveness of the proposed technique through an exhaustive set of experiments on various Computer Vision datasets (CIFAR-10, CIFAR-100, Fashion MNIST, and ImageNette), model architectures, and network topologies. Our experiments show that the proposed method achieves state-of-the-art performance for decentralized learning on heterogeneous data via a 1-6 % improvement in test accuracy compared to other existing techniques.
Bio: Aparna received her B.Tech. in Electrical Engineering from the Indian Institute of Technology, Gandhinagar, India in 2018. She joined Purdue in the Fall of 2018 as a Ph.D. student under the guidance of Professor Kaushik Roy. In the initial years of her Ph.D., she explored various topics related to efficient machine learning methods such as pruning, early exits, and low-precision training. Currently, her research interests include privacy-preserving federated learning and decentralized machine learning. In particular, she is working on developing algorithms for decentralized (peer-to-peer) learning setups which support non-IID (Non-Independent and Identically Distributed) data.