An Introduction to Optimization - Edwin K. P. Chong

. .

Résumé :

Preface xv

About the Companion Website xviii

Part I Mathematical Review 1

1 Methods of Proof and Some Notation 3

1.1 Methods of Proof 3

1.2 Notation 5

Exercises 5

2 Vector Spaces and Matrices 7

2.1 Vector and Matrix 7

2.2 Rank of a Matrix 11

2.3 Linear Equations 16

2.4 Inner Products and Norms 18

Exercises 20

3 Transformations 23

3.1 Linear Transformations 23

3.2 Eigenvalues and Eigenvectors 24

3.3 Orthogonal Projections 26

3.4 Quadratic Forms 27

3.5 Matrix Norms 32

Exercises 35

4 Concepts from Geometry 39

4.1 Line Segments 39

4.2 Hyperplanes and Linear Varieties 39

4.3 Convex Sets 41

4.4 Neighborhoods 43

4.5 Polytopes and Polyhedra 44

Exercises 45

5 Elements of Calculus 47

5.1 Sequences and Limits 47

5.2 Differentiability 52

5.3 The Derivative Matrix 54

5.4 Differentiation Rules 57

5.5 Level Sets and Gradients 58

5.6 Taylor Series 61

Exercises 65

Part II Unconstrained Optimization 67

6 Basics of Set-Constrained and Unconstrained Optimization 69

6.1 Introduction 69

6.2 Conditions for Local Minimizers 70

Exercises 78

7 One-Dimensional Search Methods 87

7.1 Introduction 87

7.2 Golden Section Search 87

7.3 Fibonacci Method 91

7.4 Bisection Method 97

7.5 Newton's Method 98

7.6 Secant Method 101

7.7 Bracketing 103

7.8 Line Search in Multidimensional Optimization 103

Exercises 105

8 Gradient Methods 109

8.1 Introduction 109

8.2 Steepest Descent Method 110

8.3 Analysis of Gradient Methods 117

Exercises 126

9 Newton's Method 133

9.1 Introduction 133

9.2 Analysis of Newton's Method 135

9.3 Levenberg-Marquardt Modification 138

9.4 Newton's Method for Nonlinear Least Squares 139

Exercises 142

10 Conjugate Direction Methods 145

10.1 Introduction 145

10.2 Conjugate Direction Algorithm 146

10.2.1 Basic Conjugate Direction Algorithm 146

10.3 Conjugate Gradient Algorithm 151

10.4 Conjugate Gradient Algorithm for Nonquadratic Problems 154

Exercises 156

11 Quasi-Newton Methods 159

11.1 Introduction 159

11.2 Approximating the Inverse Hessian 160

11.3 Rank One Correction Formula 162

11.4 DFP Algorithm 166

11.5 BFGS Algorithm 170

Exercises 173

12 Solving Linear Equations 179

12.1 Least-Squares Analysis 179

12.2 Recursive Least-Squares Algorithm 187

12.3 Solution to a Linear Equation with Minimum Norm 190

12.4 Kaczmarz's Algorithm 191

12.5 Solving Linear Equations in General 194

Exercises 201

13 Unconstrained Optimization and Neural Networks 209

13.1 Introduction 209

13.2 Single-Neuron Training 211

13.3 Backpropagation Algorithm 213

Exercises 222

14 Global Search Algorithms 225

14.1 Introduction 225

14.2 Nelder-Mead Simplex Algorithm 225

14.3 Simulated Annealing 229

14.3.1 Randomized Search 229

14.3.2 Simulated Annealing Algorithm 229

14.4 Particle Swarm Optimization 231

14.4.1 Basic PSO Algorithm 232

14.4.2 Variations 233

14.5 Genetic Algorithms 233

14.5.1 Basic Description 233

14.5.1.1 Chromosomes and Representation Schemes 234

14.5.1.2 Selection and Evolution 234

14.5.2 Analysis of Genetic Algorithms 238

14.5.3 Real-Number Genetic Algorithms 2...

Biographie:

Edwin K. P. Chong, PhD, is Professor and Head of Electrical and Computer Engineering and Professor of Mathematics at Colorado State University. He is a Fellow of the IEEE and AAAS, and was Senior Editor of the IEEE Transactions on Automatic Control.

Wu-Sheng Lu, PhD, is Professor Emeritus of Electrical and Computer Engineering at the University of Victoria, Canada. He is a Fellow of the IEEE and former Associate Editor of the IEEE Transactions on Circuits and??Systems.

Stanislaw H. ?ak, PhD, is Professor in the School of Electrical and Computer Engineering at Purdue University. He is former Associate Editor of Dynamics and Control and the IEEE Transactions on Neural Networks....

Sommaire:

Edwin K. P. Chong, PhD, is Professor and Head of Electrical and Computer Engineering and Professor of Mathematics at Colorado State University. He is a Fellow of the IEEE and AAAS, and was Senior Editor of the IEEE Transactions on Automatic Control.

Wu-Sheng Lu, PhD, is Professor Emeritus of Electrical and Computer Engineering at the University of Victoria, Canada. He is a Fellow of the IEEE and former Associate Editor of the IEEE Transactions on Circuits and??Systems.

Stanislaw H. ?ak, PhD, is Professor in the School of Electrical and Computer Engineering at Purdue University. He is former Associate Editor of Dynamics and Control and the IEEE Transactions on Neural Networks....