Task 005 - Neural Fabrics

Abstract: Novelty detection targets to automatically identify out-of-distribution (OOD) data, without any prior knowledge of them. It is a critical step in data monitoring, behavior analysis and other applications, helping enable continual learning in the field. Conventional methods of OOD detection perform multi-variate analysis on the ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical since we are incapable to anticipate the anomalies.

 

In this talk, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data, and achieves higher accuracy than previous supervised methods in all benchmarks. There are two unique contributions in our approach: (1) Gradient-based statistical analysis. Instead of using the empirical ensemble of data or features, we apply the Mahalanobis distance to the gradients that are backpropagated from the fully-connected layer in the regular classifier; (2) Assistance by a self-supervised binary classifier, which guides the label selection to generate the gradients and maximizes the Mahalanobis distance between the in-distribution and OOD data. In the comprehensive evaluation with various datasets, such as CIFAR-10, CIFAR-100, ImageNet, etc., our approach consistently outperforms state-of-the-art supervised and unsupervised methods in the Area under the Receiver Operating Characteristic (AUROC).  

 

Bio: Jingbo Sun received his B.E. degree in Communication Engineering from the Tianjin University (TJU), Tianjin, China, in 2008. He accomplished his M.S. degree in Computer Science from the University of Southern California (USC) in 2020. Prior to that, he was a Senior Hardware Engineer at Oracle, Santa Clara, U.S., where he worked on the physical design of various SPARC Microprocessors. Currently he is a Ph.D. student in Computer Engineering at Arizona State University. His research interests focus on machine learning algorithms for dynamic systems, such as novelty detection, continual learning, and graph-based perception.