First-order Methods in Optimization

The purpose of the course is to explore the theory and application of a wide range of proximal-based methods. First, we will review the basic theoretical background from convex analysis needed to understand proximal-based methods, including subgradients, conjugate functions and proximal operators. Then, in the central part of the course, we will explore several algorithms, including proximal gradient, dual proximal gradient, acceleration techniques, smoothing approaches, block decomposition variants and various splitting methods, including Lagrangian-based methods. The theoretical emphasis will be on the complexity results. On the applied side, implementation issues and applications will be discussed.

Knowledge of a first course in optimization (convexity, optimality conditions, duality…) will be assumed.