Numerical Methods for Unconstrained Optimization and Nonlinear Equations

Preface; 1. Introduction. Problems to be considered; Characteristics of 'real-world' problems; Finite-precision arithmetic and measurement of error; Exercises; 2. Nonlinear Problems in One Variable. What is not possible; Newton's method for solving one equation in one unknown; Convergence of sequences of real numbers; Convergence of Newton's method; Globally convergent methods for solving one equation in one uknown; Methods when derivatives are unavailable; Minimization of a function of one variable; Exercises; 3. Numerical Linear Algebra Background. Vector and matrix norms and orthogonality; Solving systems of linear equations—matrix factorizations; Errors in solving linear systems; Updating matrix factorizations; Eigenvalues and positive definiteness; Linear least squares; Exercises; 4. Multivariable Calculus Background; Derivatives and multivariable models; Multivariable finite-difference derivatives; Necessary and sufficient conditions for unconstrained minimization; Exercises; 5. Newton's Method for Nonlinear Equations and Unconstrained Minimization. Newton's method for systems of nonlinear equations; Local convergence of Newton's method; The Kantorovich and contractive mapping theorems; Finite-difference derivative methods for systems of nonlinear equations; Newton's method for unconstrained minimization; Finite difference derivative methods for unconstrained minimization; Exercises; 6. Globally Convergent Modifications of Newton's Method. The quasi-Newton framework; Descent directions; Line searches; The model-trust region approach; Global methods for systems of nonlinear equations; Exercises; 7. Stopping, Scaling, and Testing. Scaling; Stopping criteria; Testing; Exercises; 8. Secant Methods for Systems of Nonlinear Equations. Broyden's method; Local convergence analysis of Broyden's method; Implementation of quasi-Newton algorithms using Broyden's update; Other secant updates for nonlinear equations; Exercises; 9. Secant Methods for Unconstrained Minimization. The symmetric secant update of Powell; Symmetric positive definite secant updates; Local convergence of positive definite secant methods; Implementation of quasi-Newton algorithms using the positive definite secant update; Another convergence result for the positive definite secant method; Other secant updates for unconstrained minimization; Exercises; 10. Nonlinear Least Squares. The nonlinear least-squares problem; Gauss-Newton-type methods; Full Newton-type methods; Other considerations in solving nonlinear least-squares problems; Exercises; 11. Methods for Problems with Special Structure. The sparse finite-difference Newton method; Sparse secant methods; Deriving least-change secant updates; Analyzing least-change secant methods; Exercises; Appendix A. A Modular System of Algorithms for Unconstrained Minimization and Nonlinear Equations (by Robert Schnabel); Appendix B. Test Problems (by Robert Schnabel); References; Author Index; Subject Index.