Introduction to Optimization Methods and their Application in Statistics

1 An introduction to optimization methods.- 1.1 Introduction.- 1.2 The optimization problem.- 1.3 Some simple examples.- 1.4 Minimization procedures.- 1.5 Constrained minimization.- 1.6 Summary.- 2 Direct search methods.- 2.1 Introduction.- 2.2 Univariate search methods.- 2.3 Multiparameter search methods.- 2.4 Summary.- 3 Gradient methods.- 3.1 Introduction.- 3.2 The method of steepest descent.- 3.3 The Newton—Raphson method.- 3.4 The Davidon—Fletcher—Powell method.- 3.5 The Fletcher—Reeves method.- 3.6 Summary.- 4 Some examples of the application of optimization techniques to statistical problems.- 4.1 Introduction.- 4.2 Maximum likelihood estimation.- 4.3 Maximum likelihood estimation for incomplete data.- 4.4 Summary.- 5 Optimization in regression problems.- 5.1 Introduction.- 5.2 Regression.- 5.3 Non-linear regression.- 5.4 Log-linear and linear logistic models.- 5.5 The generalized linear model.- 5.6 Summary.- 6 Optimization in multivariate analysis.- 6.1 Introduction.- 6.2 Maximum likelihood factor analysis.- 6.3 Cluster analysis.- 6.4 Multidimensional scaling.- 6.5 Summary.- Appendix: exercises.- References.