Machine Learning and Metaheuristic Computation - Cuevas, Erik

. .

Biographie:

About the Authors xi

Preface xiii

Acknowledgments xvii

Introduction xix

1 Fundamentals of Machine Learning 1

1.1 Introduction 1

1.2 Different Types of Machine Learning Approaches 4

1.3 Supervised Learning 6

1.4 Unsupervised Learning 8

1.5 Reinforcement Learning 10

1.6 Which Algorithm to Apply? 13

1.7 Recommendation to Build a Machine Learning Model 15

References 19

2 Introduction to Metaheuristics Methods 21

2.1 Introduction 21

2.2 Classic Optimization Methods 23

2.3 Descending Gradient Method 24

2.4 Metaheuristic Methods 29

2.5 Exploitation and Exploration 35

2.6 Acceptance and Probabilistic Selection 37

2.7 Random Search 41

2.8 Simulated Annealing 47

References 57

3 Fundamental Machine Learning Methods 59

3.1 Introduction 59

3.2 Regression 60

3.2.1 Explanatory Purpose 62

3.2.2 Predictive Purpose 62

3.3 Classification 71

3.3.1 Relationship Between Regression and Classification 72

3.3.2 Differences Between Regression and Classification 72

3.4 Decision Trees 73

3.4.1 Procedure of Classification 74

3.4.2 Determination of the Splitting Point 77

3.4.2.1 Gini Index 77

3.4.2.2 Entropy 78

3.4.3 Example of Classification 79

3.5 Bayesian Classification 86

3.5.1 Conditional Probability 87

3.5.2 Classification of Fraudulent Financial Reports 87

3.5.3 Practical Constraints by Using the Exact Bayes Method 90

3.5.4 Naive Bayes Method 90

3.5.5 Computational Experiment 92

3.6 k-Nearest Neighbors (k-NN) 99

3.6.1 k-NN for Classification 99

3.6.2 k-NN for Regression 101

3.7 Clustering 105

3.7.1 Similarity Indexes 107

3.7.2 Methods for Clustering 108

3.8 Hierarchical Clustering 112

3.8.1 Implementation in MATLAB 114

3.9 K-Means Algorithm 122

3.9.1 Implementation of K-Means Method in MATLAB 127

3.10 Expectation-Maximization Method 130

3.10.1 Gaussian Mixture Models 131

3.10.2 Maximum Likelihood Estimation 131

3.10.3 EM in One Dimension 132

3.10.3.1 Initialization 132

3.10.3.2 Expectation 132

3.10.3.3 Maximization 133

3.10.4 Numerical Example 133

3.10.5 EM in Several Dimensions 135

References 141

4 Main Metaheuristic Techniques 145

4.1 Introduction 145

4.1.1 Use of Metaphors 145

4.1.2 Problems of the Use of Metaphors 146

4.1.3 Metaheuristic Algorithms 147

4.2 Genetic Algorithms 148

4.2.1 Canonical Genetic Algorithm 149

4.2.2 Selection Process 152

4.2.3 Binary Crossover Process 155

4.2.4 Binary Mutation Process 156

4.2.5 Implementation of the Binary GA 157

4.2.6 Genetic Algorithm Utilizing Real-Valued Parameters 164

4.2.7 Crossover Operator for Real-Valued Parameters 165

4.2.8 Mutation Operator for Real-Valued Parameters 176

4.2.9 Computational Implementation of the GA with Real Parameters 181

4.3 Particle Swarm Optimization (PSO) 189

4.3.1 Strategy for Searching in Particle Swarm Optimization 189

4.3.2 Analysis of the PSO Algorithm 192

4.3.3 Inertia Weighting 192

4.3.4 Particle Swarm Optimization Algorithm Using MATLAB 193

4.4 Differential Evolution (DE) Algorithm 196

4.4.1 The Search Strategy of DE 197

4.4.2 The Mutation Operation in DE 200

4.4.2.1 Mutation Rand/ 1 201

4.4.2.2 Mutaci?n Best/ 1 201

4.4.2.3 Mutation Rand/ 2 202

4.4.2.4 Mutation Best/ 2 202

4.4.2.5 Mutation Current-to-Best/ 1 203

4....