brain iconTheme 1: Neuro-inspired Algorithms and Theory

Despite the spectacular success of machine learning, especially deep neural networks, in some application domains, such as its use in next generation autonomous intelligent systems, key challenges remain. Fig. 1.1 summarizes these challenges and the advances that we will pursue in Theme 1 through neuro-inspired algorithms and theory.

 

State-of-the-Art:
Deep Neural Nets

  • Largely supervised learning
  • Static (one-time) learning
  • Training requires global updates (Backpropagation / SGD)
  • Perception (speech, images, text)
  • Empirical validation
  • Manually designed network topologies

 

Future Neuro-Inspired Algorithms

  • Unsupervised or minimally supervised learning
  • Incremental and lifelong learning
  • Localized training (beyond SGD and STDP)
  • Perception, reasoning and decision making
  • Theoretical analysis and bounds
  • Biologically informed network topologies

Fig. 1.1: Key challenges and goals of Theme 1

Current learning algorithms require unsustainably large amounts of training data and computation as well as considerable manual effort to design the network topology. The success of current deep neural nets is based largely on perception tasks, whereas autonomous intelligent systems will require them to perform not only perception, but also reasoning and decision-making. Mission-critical applications will require high levels of robustness and verifiability of generalization behavior. Emerging hardware substrates based on approximate and stochastic hardware and post-CMOS devices do not directly lend themselves to executing current algorithms.

To address these challenges, we will perform the following tasks: 1.1: Neuro-inspired algorithms for efficient and lifelong learning; 1.2: Theoretical underpinnings of neuro-inspired computing, and 1.3: Algorithms for emerging hardware.

Theme Leader

Principal Investigators (PIs)