Surrogate Modeling and Optimization - Kim, Nam-Ho

. .

Résumé :

Preface xiii

Acknowledgment xvii

About the Companion Website xix

Part I Basics of Surrogate Modeling 1

1 Introduction to Surrogate Models 3

1.1 What Is Surrogate Modeling? 3

1.2 Surrogate Models 5

1.3 Design of Experiments: Sampling 7

1.4 Interpolation Versus Extrapolation 8

1.5 Flowchart of Surrogate Modeling 10

1.6 Overview of Surrogate Modeling 12

1.7 Smoothness and Loss Function 14

2 Polynomial Response Surfaces 17

2.1 Introduction 17

2.2 Curve Fitting 19

2.3 Linear Regression 23

2.3.1 Polynomial Response Surface 23

2.3.2 Polynomial Response Surface in Multiple Dimensions 28

2.3.3 Curse of Dimensionality 30

2.3.4 Assumptions in Linear Regression 32

2.4 Goodness of Fit 33

2.4.1 Estimation of Noise in Samples 34

2.4.2 Coefficient of Multiple Determination 38

2.4.3 Cross-validation 43

2.5 Confidence of Coefficients and Backward Elimination 48

2.6 Prediction Variance 51

2.6.1 Prediction Uncertainty 52

2.6.2 Sample Sensitivity 53

2.6.3 Prediction Variance with Variable Noise 55

2.7 Outliers 57

2.8 Statistical View of Linear Regression 59

Exercise 65

3 Design of Experiments 71

3.1 Introduction 71

3.2 Design of Experiments in Box-like Domains 73

3.2.1 Scaling of Input Variables 73

3.2.2 Interpolation, Extrapolation, and Prediction Variance 75

3.2.3 Designs for Linear Polynomial Response Surfaces 79

3.2.4 Designs for Quadratic Polynomial Response Surfaces 80

3.3 Optimal Design of Experiments 90

3.3.1 D-Optimal Design 91

3.3.2 A-Optimal Design 95

3.3.3 G-Optimal Design 96

3.3.4 Minimum Bias Design 99

3.4 Space-Filling Design of Experiments 104

3.4.1 Monte Carlo Simulation 104

3.4.2 Latin Hypercube Sampling 105

3.4.3 Orthogonal Arrays 109

3.5 Review of Various Designs of Experiments 111

3.5.1 Guideline for Selecting Designs of Experiments 111

3.5.2 Good Practice for Design of Experiments 112

Exercise 113

Part II Design Optimization 117

4 Optimization Definition and Formulation 119

4.1 Introduction 119

4.2 Design Optimization Definition 120

4.2.1 Design Optimization Process 120

4.2.2 Design Variables and Feasible Domain 122

4.2.3 Graphical Optimization 126

4.3 Optimization Problem Formulation 128

4.3.1 Three-step Problem Definition 128

4.3.2 Standard Form 129

4.3.3 Normalization 130

4.3.4 Convex Function and Convex Problem 133

4.4 Optimality Criteria 135

4.4.1 Global Versus Local Optimum 135

4.4.2 Unconstrained Optimization 136

4.4.3 Constrained Optimization 141

4.4.4 Effect of Constraint Limit 149

4.4.5 Sensitivity of Optimum Solution to Parameters 151

Exercise 153

5 Numerical Optimization Algorithms 161

5.1 Introduction 161

5.2 Overview of the Numerical Optimization Process 162

5.3 Determination of Step Size 164

5.3.1 Descent Direction 164

5.3.2 Step-Size Termination Criterion 165

5.3.3 Interval Reduction Method 166

5.3.4 Quadratic Interpolation Method 167

5.4 Unconstrained Optimization Algorithms 168

5.4.1 Steepest Descent Method 169

5.4.2 Conjugate Gradient Method 171

5.4.3 Newton Method 173

5.4.4 Quasi-Newton Method 174

5.4.5 Rate of Convergence 177

5.5 Constrained Optimization Using Unconstrained Algorithms 178

5.5.1 Lagrange Multiplier Method 179

5.5.2 Penalty Function Method 180

Biographie:

Nam-Ho Kim is a Professor in the Department of Mechanical and Aerospace Engineering at the University of Florida. His research interests include design under uncertainty, prognostics and health management, verification validation and uncertainty quantification, and nonlinear structural mechanics. He has more than twenty years of experience teaching materials in these fields to graduate students....

Sommaire:

Preface xiii
Acknowledgment xvii
About the Companion Website xix

Part I Basics of Surrogate Modeling 1

1 Introduction to Surrogate Models 3
1.1 What Is Surrogate Modeling? 3
1.2 Surrogate Models 5
1.3 Design of Experiments: Sampling 7
1.4 Interpolation Versus Extrapolation 8
1.5 Flowchart of Surrogate Modeling 10
1.6 Overview of Surrogate Modeling 12
1.7 Smoothness and Loss Function 14

2 Polynomial Response Surfaces 17
2.1 Introduction 17
2.2 Curve Fitting 19
2.3 Linear Regression 23
2.4 Goodness of Fit 33
2.5 Confidence of Coefficients and Backward Elimination 48
2.6 Prediction Variance 51
2.7 Outliers 57
2.8 Statistical View of Linear Regression 59

3 Design of Experiments 71
3.1 Introduction 71
3.2 Design of Experiments in Box-like Domains 73
3.3 Optimal Design of Experiments 90
3.4 Space-Filling Design of Experiments 104
3.5 Review of Various Designs of Experiments 111

Part II Design Optimization 117

4 Optimization Definition and Formulation 119
4.1 Introduction 119
4.2 Design Optimization Definition 120
4.3 Optimization Problem Formulation 128
4.4 Optimality Criteria 135

5 Numerical Optimization Algorithms 161
5.1 Introduction 161
5.2 Overview of the Numerical Optimization Process 162
5.3 Determination of Step Size 164
5.4 Unconstrained Optimization Algorithms 168
5.5 Constrained Optimization Using Unconstrained Algorithms 178
5.6 Constrained Optimization Using Direct Methods 182
5.7 Matlab Optimization Toolbox 187
5.8 Practical Suggestions for Numerical Optimization 191

6 Global Search Optimization Algorithms 197
6.1 Introduction 197
6.2 Nelder-Mead Sequential Simplex Algorithm 198
6.3 DIRECT Method 202
6.4 Genetic Algorithms 208
6.5 Particle Swarm Optimization 217
6.6 Simulated Annealing Optimization 221

Part III Advanced Topics in Surrogate Modeling 227

7 Kriging Surrogate-Gaussian Process Model 229
7.1 Introduction 229
7.2 Kriging Philosophy 230
7.3 Kriging Surrogate Model 238
7.4 Issues in Determining Hyperparameters 252
7.5 Numerical Implementation of Kriging Surrogate 270
7.6 Kriging with Nuggets-Fitting with Noisy Data (Gaussian Process Regression) 282

8 Neural Network Model 295
8.1 Introduction 295
8.2 Feedforward Neural Network Model 297
8.3 Matlab Functions for Feedforward Neural Network 310
8.4 Uncertainty Quantification in Neural Network Models 316
8.5 Issues in Feedforward Neural Network 327
8.6 Neural Networks with Constraints 340

9 Multi-fidelity Surrogate Models 353
9.1 Introduction 353
9.2 Multi-fidelity Surrogate Models 356
9.3 Regression-based Multi-fidelity Surrogate 362
9.4 Kriging-based Multi-fidelity Surrogate 369
9.5 Sampling Strategy for Multi-fidelity Surrogate Modeling 386
9.6 Challenges and Recommendations 392

10 Efficient Global Optimization 397
10.1 Introduction 397
10.2 Efficient Global Optimization 399
10.3 Efficient Global Optimization Using Polynomial Response Surface 413
10.4 Efficient Global Optimization Using Kriging Surrogate 421

Exercise 427
References 429
Index 435