Abstract
We attempt to unveil the working principle of simple convolutional neural networks (CNNs) through a constructive, feedforward and interpretable design in this work. A CNN is simple if it is a cascade of two networks, where the first one consists of convolutional layers and the second one contains fully connected layers. They are called the Conv_net and the FC_net, respectively. To begin with, we develop a new signal transform, called the Saab (Subspace Approximation with adjusted bias) transform, and use it to design the Conv_net.  The bias term in the Saab transform is chosen to annihilate the effect of the REctified Linear Unit (ReLU). As a result, nonlinearity due to the activation function can be eliminated. Next, we propose a label-guided linear least squared regression (L3SR) method in the design of the FC_net. Furthermore, we introduce a cross-entropy loss function to analyze the behavior of the FC_net. Finally, extensive experiments are conducted to illustrate the feedforward design methodology and validate the quality of the proposed designs in their classification performance.

Bio
Dr. C.-C. Jay Kuo received his Ph.D. degree from the Massachusetts Institute of Technology in 1987. He is now with the University of Southern California (USC) as Director of the Media Communications Laboratory and Distinguished Professor of Electrical Engineering and Computer Science. His research interests are in the areas of media processing, compression and understanding. Dr. Kuo was the Editor-in-Chief for the IEEE Trans. on Information Forensics and Security in 2012-2014. He was the Editor-in-Chief for the Journal of Visual Communication and Image Representation in 1997-2011, and served as Editor for 10 other international journals. Dr. Kuo received the 1992 National Science Foundation Young Investigator (NYI) Award, the 1993 National Science Foundation Presidential Faculty Fellow (PFF) Award, the 2010 Electronic Imaging Scientist of the Year Award, the 2010-11 Fulbright-Nokia Distinguished Chair in Information and Communications Technologies, the 2011 Pan Wen-Yuan Outstanding Research Award, the 2014 USC Northrop Grumman Excellence in Teaching Award, the 2016 USC Associates Award for Excellence in Teaching, the 2016 IEEE Computer Society Taylor L. Booth Education Award, the 2016 IEEE Circuits and Systems Society John Choma Education Award, the 2016 IS&T Raymond C. Bowman Award, and the 2017 IEEE Leon K. Kirchmayer Graduate Teaching Award. Dr. Kuo is a Fellow of AAAS, IEEE and SPIE. He has guided 145 students to their Ph.D. degrees and supervised 27 postdoctoral research fellows. Dr. Kuo is a co-author of 260 journal papers, 900 conference papers and 14 books.