Course Title: Convex Optimization – 3 credit hours

Course Description: The course concentrates on developing skills to recognize and solve convex optimization problems. Topics to be covered are the basic theory of optimization, convex sets, convex functions, convex analysis, least-squares, linear and quadratic programs, semidefinite programming, minimax, optimality conditions and the duality theorem. Methods for solving optimization problems including steepest descent, conjugate gradient, and interior point methods are discussed in details.

Course Objectives: Optimization is an essential tool for students in science and engineering programs and many other disciplines including economy, finance, and operation research. The main objective of this course is to provide students an opportunity to learn the optimization theory based on the modern approach of Convex optimization. Students will receive training to cast problems in a form of convex programming and use iterative computer based techniques to solve the problem and find optimal solutions of different problems. Many applications of the methods in signal processing, machine learning, data analysis, market evaluation and making optimal predictions or estimations will be discussed during lectures.

In summary, the course has three main objectives:

1. Recognize and formulate convex optimization problems that arise in applications,
2. Analyze a convex problem using convexity theory and duality theory,
3. Understand how to solve convex problems using numerical techniques and obtain some practice in solving them.

Main textbook: Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004. PDF of the textbook is available for free by the author.

Instructor’s name	Ramin Pashaie
Office address	EE 325
Office Hours	2 hours a week (2 sessions)
Contact telephone number	561-297-1041
Email address	rpashaie@fau.edu

Course topical outline, including dates for exams/quizzes, papers, completion of reading:

Week – 1 Introduction to optimization

• The Role of optimization, convexity
• Examples of application (communications, signal processing)

Week – 2 Review of linear algebra and mathematics background,

Week – 3,4 Convex set and convex function,

• Convex set, convex functions
• Operations that preserve convexity (both sets and functions)
• Conjugate function, conjugate sets
• Separating hyper-plane theorem

Week – 5,6 Convex optimization problems

• Optimization problem definition and examples,
• Linear programming,
• Quadratic programming,
• Geometric programming,
• Semi-definite programming.

Week – 7,8 Duality,

• Lagrangian dual function (conjugate function)
• Lagrange dual problem
  1. Properties, weak and strong duality
  2. Interpretation of dual variables, duality (geometric, saddle point, economics)
• Optimality conditions
  1. KKT, necessity and sufficiency
  2. Sub-gradients for non-smooth functions
• Examples:
  1. Water-filling and reverse water-filling
  2. Multiple-access sum capacity (scalar or MIMO version)
  3. Compress sensing (using sub-gradient)

Week – 9-11 Methods and algorithms

• Unconstrained
  1. Gradient descent, steepest descent
  2. Newton method
• With equality constraints
  1. Newton methods with equality constraints
  2. ADMM method
  3. Sub-gradient method
• With inequality constraints
  1. Barrier interior point method
  2. Primal-dual interior point methods

Week – 12,13 Advanced topics,

• First-order methods for large-scale optimization,
  1. First-order gradient descent,
  2. Application in machine learning.
• Schur convexity

Week -14,15 Solving Techniques

• Steepest Descent
• Conjugate Gradient
• Interior point method