Optimization Methods for Systems and Control
ECE58000
Credit Hours:
3Learning Objective:
- Use various methods to compute minimum or maximum of nonlinear functions of many variables
- Solve linear programming problems
- Find minimum or maximum of nonlinear functions of many variables using population-based methods
- Apply various optimization methods learned in the course to real-life design problems
Description:
This course provides an introduction to various methods of obtaining the extremum (minimum or maximum) of a non-dynamical system and the use of these methods in real-life applications. Computational methods for nonlinear optimization; unconstrained optimization. Constrained optimization; linear programming; simplex method for solving linear programs; Lagrange's conditions, the Karush-Kuhn-Tucker (KKT) conditions, Least squares, Convex optimization, Global optimization methods: Genetic algorithms and Particle swarm optimization (PSO) method.
Topics Covered:
| Week | Modules |
|---|---|
| 1 |
1. Welcome, Introduction, and Motivation 2. Differentiation of functions of many variables 3. Taylor series expansion and feasible directions |
| 2 |
4. First-order necessary condition (FONC) 5. Necessary and sufficient conditions for a point to be a minimizer 6. Quadratic forms |
| 3 |
7. Quadratic forms and second-order sufficient condition (SOSC) 8. Proof of the second-order sufficient condition (SOSC) 9. Golden section search and Fibonacci method |
| 4 |
10. The Fibonacci line search method and Newton's methods 11. Introduction to the method of steepest descent (SD) 12. Performance analysis of the steepest descent (SD) gradient method |
| 5 |
13. Further analysis of the steepest descent (SD) gradient method 14. Newton's method to minimize functions of many variables 15. The asymptomatic symbols and the conjugate directions (CD) method |
| 6 |
16. Linear independence of conjugate directions 17. Implementation of the Conjugate Gradient (CG) algorithm 18. Review for Midterm 1 |
| 7 |
19. An introduction to quasi-Newton methods 20. The rank one correction of Davidon-Fletcher-Powell methods 21. The Broyden-Fletcher-Goldfarb-Shanno algo |
| 8 |
22. The least-squares method development and its applications 23. The least squares and the recursive least-squares (RLS) methods 24. Underdetermined systems of linear equations |
| 9 |
25. Solving a general system of linear equations 26. The pseudo-inverse and population based optimization methods 27. Particle Swarm Optimization and Canonical Genetic Algorithms |
| 10 |
28. The canonical genetic algorithm (GA) development 29. The canonical genetic algorithm implementation and analysis 30. The canonical genetic algorithm MATLAB implementation |
| 11 |
31. The schema theorem and the real-number genetic algorithms 32. Real-number genetic algorithms and linear programming 33. Fundamental theorem of linear programming |
| 12 |
34. Fundamental theorem of linear programming and convex sets 35. Basic feasible solutions and extreme points of the feasible set |
| 13 |
36. Simplex method for solving linear programming problems 37. The two-phase simplex method and and introduction to duality 38. An introduction to nonlinear optimization subject to constraints |
| 14 |
39. Nonlinear optimization subject to equality constraints 40. The Lagrange multiplier theorem 41. Nonlinear optimization subject to inequality constraints |
| 15 |
42. Convex functions and convex optimization problem |
Prerequisites:
- MA 511 or equivalent (first graduate course in linear algebra)
- Linear algebra, calculus of several variables. In particular: matrix manipulation, linear spaces, quadratic forms, tangent planes. Elements of multivariable calculus, in particular, differentiation of real-valued functions of n variables, gradients, and the chain rule.