Many modern statistical estimation problems are defined by  three major components: a statistical model that postulates the dependence of an output variable on the input features; a loss function measuring the error between the observed output and the model predicted output; and a regularizer that controls the overfitting and/or variable selection in the model.  We study the sampled version of this generic statistical estimation problem where the model parameters are estimated by empirical risk minimization, which involves the minimization of the empirical average of the loss function at the data points weighted by the model regularizer.  In our setup we allow all three component functions to be of the difference-of-convex (dc) type and illustrate them with a host of commonly used examples, including those in continuous piecewise affine regression and in deep learning with piecewise affine activation functions.  We describe a non-monotone majorization-minimization (MM) algorithm for solving the unified nonconvex, non-differentiable optimization problem which is formulated as a specially structured composite dc program of the pointwise max type, and present convergence results to a directional stationary solution.  An efficient semi smooth Newton method is proposed to solve the dual of the MM sub-problems.  Numerical results are presented to demonstrate the effectiveness of the proposed algorithm and the superiority of continuous piecewise affine regression over the standard linear model. 

Dr. Jong-Shi Pang joined the University of Southern California as the Epstein Family Professor of Industrial and Systems Engineering in August 2013. Prior to this position, he was the Caterpillar Professor and Head of the Department of Industrial and Enterprise Systems Engineering for six years between 2007 and 2013. He held the position of the Margaret A. Darrin Distinguished Professor in Applied Mathematics in the Department of Mathematical Sciences and was a Professor of Decision Sciences and Engineering Systems at Rensselaer Polytechnic Institute from 2003 to 2007. He was a Professor in the Department of Mathematical Sciences at the Johns Hopkins University from 1987 to 2003), he was a Program Director in the Division of Mathematical Sciences at the National Science Foundation.

  

Professor Pang was a winner of the 2003 George B. Dantzig Prize awarded jointly by the Mathematical Programming Society and the Society for Industrial and Applied Mathematics for his work on finite-dimensional variational inequalities, and a co-winner of the 1994 Frederick W. Lanchester Prize awarded by the Institute for Operations Research and Management Science. Several of his publications have received best paper awards in different engineering fields: signal processing, energy and natural resources, computational management science, and robotics and automation.  Dr. Pang is a member in the inaugural 2009 class of Fellows of the Society for Industrial and Applied Mathematics. Professor Pang's general research interest is in the mathematical modeling and analysis of a wide range of complex engineering and economics systems with focus in operations research, (single and multi-agent) optimization, equilibrium programming, and constrained dynamical systems.