We will discuss modern computational methods for: finding a minimum
or maximum of a nonlinear function of one or more variables, solving systems
of nonlinear equations, solving nonlinear least squares ("data fitting",
"regression") problems. Our aim is to enable the student to understand
and develop algorithms for these and related problems.
The problems we consider occur in a myriad of scientific, engineering, and business applications, often in cases when the nonlinear function involved is itself an expensive computational or experimental procedure or the number of parameters is large. Therefore we will emphasize the efficiency of our algorithms. We will concentrate on the areas of practical difficulty, including convergence from bad starting points, and what to do when analytic derivatives are unavailable.
We will give some special attention this semester to methods for solving large scale problems, and to real-world applications.
Prerequisites: Undergraduate linear algebra and vector analysis, computer programming experience. Some numerical analysis helpful but not required.
Course Outline:
I. Sample problems.
II. Methods for problems in one variable.
III. Background: Linear algebra, vector calculus.
IV. Newton's method for unconstrained optimization and
systems of equations.
V. Global methods: line searches, trust regions.
VI. Methods when derivatives are unavailable ("quasi-Newton"
methods).
VII. Nonlinear least squares.
VIII. Methods for large scale problems.
Text: Numerical Methods for Unconstrained Optimization and Nonlinear Equations by Dennis and Schnabel (SIAM, 1996)
Workload: As appropriate for a seminar-level course, including computer projects, homework assignments, independent reading in the literature. No exams.
Inquiries welcome by email to bobby@cs.colorado.edu