Abstract: Distributed and large-scale optimization problems have gained a significant attention in several engineering areas, including network information processing, communication networks, cyber-physical systems, multi-agent control, and machine learning, just to name a few. The large-scale property is reflected in the number of decision variables, the number of constraints, or both, while the distributed nature of the problems is inherent due to partial (local) knowledge of the problem data (e.g., a portion of the cost function or a subset of the constraints is known to different entities in the system). Moreover, many applications of interest lead to optimization problems with nonconvex objective and constraints. All this makes the analysis and design of parallel and distributed algorithms a challenging task.

In this talk we survey various recent parallel, distributed, asynchronous algorithms for the aforementioned classes of nonconvex problems. We show that several existing schemes can be unified under the elegant umbrella of successive convex approximation methods. The proposed unified algorithmic framework is then tested on a variety of applications in signal processing, communications, and machine learning. Finally, we address the question on what rigorous guarantees these methods provide for various classes of nonconvex functions, thus shifting the barrier between tractable and intractable problems.