CSCI5254: Convex Optimization and Its Applications

Course Description

This course discusses basic convex analysis (convex sets, functions, and optimization problems), optimization theory (linear, quadratic, semidefinite, and geometric programming; optimality conditions and duality theory), some optimization algorithms (descent methods and interior-point methods), basic applications (in signal processing, machine learning, statistics, control, communications, networks, circuit design, mechanical engineering, and finance, etc), and some advanced topics (distributed decomposition, exact convex relaxation, and parsimonious recovery, if time permits). The objective is to give students the theoretical tools and training to recognize and formulate convex optimization problems that arise in engineering and applied science as well as the understanding of how such problems are solved, and to give students the background required to use the tools and methods of optimization in their own applications and researches.

Time and Location

WM 1:25PM - 2:40PM; DLC 1B20
Instructor
Lijun Chen (lijun.chen@colorado.edu)
Office hours: TUE 3:00PM - 5:00PM, by Zoom, Starting August 31
Textbook
S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. (This book is available on-line, but I strongly recommend to buy a hardcopy).
Optional or Other References
D. Bertsekas, A. Nedic, and A. Ozdaglar, Convex Analysis and Optimization, Athena Scientific, 2003.
Prerequisites
The Appendix A of the textbook by Boyd and Vandenberghe provides a comprehensive review of the required mathematical background.


Lectures
This is a tenetaive outline of the topics we will cover and will likely change as the semester goes by.
  • Introduction (Aug 23)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 1.

  • Convex sets (Aug 25 & 30)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 2.

  • Convex Functions (Sep 1 & 8)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 3.

  • Convex Optimization Problems (Sep 13, 15 & 20)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 4.

  • Duality (Oct 22, 27 & 29)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 5.1-5.7, 5.9.

  • Approximation and Fitting (Oct 4 & 6)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 6.

  • Statistical Estimation (Oct 11 & 13)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 7.

  • Geometric Problems (Oct 18 & 20)
    Lecture Notes
    Reading: Boyd and Vandenberghe, Chapter 8.

  • Algorithms (Oct 25, 27, Nov 1, 3, 8, 10 & 15)
    Lecture Notes(Unconstrained problems)
    Lecture Notes(Equality constrained problems)
    Lecture Notes(Interior-point methods)
    Reading: Boyd and Vandenberghe, Chapter 9, 10, 11.

  • Advanced Topics (Stochastic gradient, reinforcement learning, etc; Oct 17, 29, Dec 1, 6 & 8)
    Lecture Notes

Grading
This is a preliminary breakdown that may change during the semester.
  • Homework -- 40%
  • Final -- 50%
  • Participation -- 10%
Homeworks
There will be about eight to ten homework sets, due in class on Thursdays. Students are strongly encouraged to collaborate on homeworks, but should make sure to fully understand the problems and solutions and must write down solutions independently.
Final
There will be a 24-hour take home final exam.

Acknowledgement
  • The lecture slides are adopted from Dr. Stephen Boyd's letcture notes on Convex Optimization at Standord University. Thank Dr. Boyd for his kind help and support.


Classroom Behavior
Both students and faculty are responsible for maintaining an appropriate learning environment in all instructional settings, whether in person, remote or online. Those who fail to adhere to such behavioral standards may be subject to discipline. Professional courtesy and sensitivity are especially important with respect to individuals and topics dealing with race, color, national origin, sex, pregnancy, age, disability, creed, religion, sexual orientation, gender identity, gender expression, veteran status, political affiliation or political philosophy. For more information, see the policies on classroom behavior and the Student Conduct & Conflict Resolution policies.
Requirements for COVID-19
As a matter of public health and safety due to the pandemic, all members of the CU Boulder community and all visitors to campus must follow university, department and building requirements and all public health orders in place to reduce the risk of spreading infectious disease. Students who fail to adhere to these requirements will be asked to leave class, and students who do not leave class when asked or who refuse to comply with these requirements will be referred to the Student Conduct & Conflict Resolution policies. For more information, see the policy on classroom behavior and the Student Code of Conduct. If you require accommodation because a disability prevents you from fulfilling these safety measures, please follow the steps in the “Accommodation for Disabilities” statement on this syllabus. As of Aug. 13, 2021, CU Boulder has returned to requiring masks in classrooms and laboratories regardless of vaccination status. This requirement is a temporary precaution during the delta surge to supplement CU Boulder’s COVID-19 vaccine requirement. Exemptions include individuals who cannot medically tolerate a face covering, as well as those who are hearing-impaired or otherwise disabled or who are communicating with someone who is hearing-impaired or otherwise disabled and where the ability to see the mouth is essential to communication. If you qualify for a mask-related accommodation, please follow the steps in the “Accommodation for Disabilities” statement on this syllabus. In addition, vaccinated instructional faculty who are engaged in an indoor instructional activity and are separated by at least 6 feet from the nearest person are exempt from wearing masks if they so choose. Students who have tested positive for COVID-19, have symptoms of COVID-19, or have had close contact with someone who has tested positive for or had symptoms of COVID-19 must stay home. In this class, if you are sick or quarantined, nnotify the instructor in time to discuss ways to accommdate the situation.
Accommodations for Disability
If you qualify for accommodations because of a disability, please submit your accommodation letter from Disability Services to your faculty member in a timely manner so that your needs can be addressed. Disability Services determines accommodations based on documented disabilities in the academic environment. Information on requesting accommodations is located on the Disability Services website. Contact Disability Services at 303-492-8671 or dsinfo@colorado.edu for further assistance. If you have a temporary medical condition, see Temporary Medical Conditions on the Disability Services website.
Religious Observances
Campus policy regarding religious observances requires that faculty make every effort to deal reasonably and fairly with all students who, because of religious obligations, have conflicts with scheduled exams, assignments or required attendance. In this class, we accommodate the absences of the class and delay in homeworks and project due to religious observances, upon notifying the instructor one week ahead. See campus policy regarding religious observances for full details.
Discrimination and Harassment
The University of Colorado Boulder (CU-Boulder) is committed to maintaining a positive learning, working, and living environment. The University of Colorado does not discriminate on the basis of race, color, national origin, sex, age, disability, creed, religion, sexual orientation, or veteran status in admission and access to, and treatment and employment in, its educational programs and activities. (Regent Law, Article 10, amended 11/8/2001). CU-Boulder will not tolerate acts of discrimination or harassment based upon Protected Classes or related retaliation against or by any employee or student. For purposes of this CU-Boulder policy, "Protected Classes" refers to race, color, national origin, sex, pregnancy, age, disability, creed, religion, sexual orientation, gender identity, gender expression, or veteran status. Individuals who believe they have been discriminated against should contact the Office of Discrimination and Harassment (ODH) at 303-492-2127 or the Office of Student Conduct (OSC) at 303-492-5550. Information about the ODH, the above referenced policies, and the campus resources available to assist individuals regarding discrimination or harassment can be obtained at http://www.colorado.edu/odh.
Honor Code
All students of the University of Colorado at Boulder are responsible for knowing and adhering to the academic integrity policy of this institution. Violations of this policy may include: cheating, plagiarism, aid of academic dishonesty, fabrication, lying, bribery, and threatening behavior. All incidents of academic misconduct shall be reported to the Honor Code Council (honor@colorado.edu; 303-735-2273). Students who are found to be in violation of the academic integrity policy will be subject to both academic sanctions from the faculty member and non-academic sanctions (including but not limited to university probation, suspension, or expulsion). Other information on the Honor Code can be found at http://www.colorado.edu/policies/honor.html and at http://www.colorado.edu/academics/honorcode/.


drupal stats