Machine Learning Theory and Applications - Xavier Vasques

. .

Résumé :

Foreword xiii

Acknowledgments xv

General Introduction xvii

1 Concepts, Libraries, and Essential Tools in Machine Learning and Deep Learning 1

1.1 Learning Styles for Machine Learning 2

1.1.1 Supervised Learning 2

1.1.1.1 Overfitting and Underfitting 3

1.1.1.2 K-Folds Cross-Validation 4

1.1.1.3 Train/Test Split 4

1.1.1.4 Confusion Matrix 5

1.1.1.5 Loss Functions 7

1.1.2 Unsupervised Learning 9

1.1.3 Semi-Supervised Learning 9

1.1.4 Reinforcement Learning 9

1.2 Essential Python Tools for Machine Learning 9

1.2.1 Data Manipulation with Python 10

1.2.2 Python Machine Learning Libraries 10

1.2.2.1 Scikit-learn 10

1.2.2.2 TensorFlow 10

1.2.2.3 Keras 12

1.2.2.4 PyTorch 12

1.2.3 Jupyter Notebook and JupyterLab 13

1.3 HephAIstos for Running Machine Learning on CPUs, GPUs, and QPUs 13

1.3.1 Installation 13

1.3.2 HephAIstos Function 15

1.4 Where to Find the Datasets and Code Examples 32

Further Reading 33

2 Feature Engineering Techniques in Machine Learning 35

2.1 Feature Rescaling: Structured Continuous Numeric Data 36

2.1.1 Data Transformation 37

2.1.1.1 StandardScaler 37

2.1.1.2 MinMaxScaler 39

2.1.1.3 MaxAbsScaler 40

2.1.1.4 RobustScaler 40

2.1.1.5 Normalizer: Unit Vector Normalization 42

2.1.1.6 Other Options 43

2.1.1.7 Transformation to Improve Normal Distribution 44

2.1.1.8 Quantile Transformation 48

2.1.2 Example: Rescaling Applied to an SVM Model 50

2.2 Strategies to Work with Categorical (Discrete) Data 57

2.2.1 Ordinal Encoding 59

2.2.2 One-Hot Encoding 61

2.2.3 Label Encoding 62

2.2.4 Helmert Encoding 63

2.2.5 Binary Encoding 64

2.2.6 Frequency Encoding 65

2.2.7 Mean Encoding 66

2.2.8 Sum Encoding 68

2.2.9 Weight of Evidence Encoding 68

2.2.10 Probability Ratio Encoding 70

2.2.11 Hashing Encoding 71

2.2.12 Backward Difference Encoding 72

2.2.13 Leave-One-Out Encoding 73

2.2.14 James-Stein Encoding 74

2.2.15 M-Estimator Encoding 76

2.2.16 Using HephAIstos to Encode Categorical Data 77

2.3 Time-Related Features Engineering 77

2.3.1 Date-Related Features 79

2.3.2 Lag Variables 79

2.3.3 Rolling Window Feature 82

2.3.4 Expending Window Feature 84

2.3.5 Understanding Time Series Data in Context 85

2.4 Handling Missing Values in Machine Learning 88

2.4.1 Row or Column Removal 89

2.4.2 Statistical Imputation: Mean, Median, and Mode 90

2.4.3 Linear Interpolation 91

2.4.4 Multivariate Imputation by Chained Equation Imputation 92

2.4.5 KNN Imputation 93

2.5 Feature Extraction and Selection 97

2.5.1 Feature Extraction 97

2.5.1.1 Principal Component Analysis 98

2.5.1.2 Independent Component Analysis 102

2.5.1.3 Linear Discriminant Analysis 110

2.5.1.4 Locally Linear Embedding 115

2.5.1.5 The t-Distributed Stochastic Neighbor Embedding Technique 123

2.5.1.6 More Manifold Learning Techniques 125

2.5.1.7 Feature Extraction with HephAIstos 130

2.5.2 Feature Selection 131

2.5.2.1 Filter Methods 132

2.5.2.2 Wrapper Methods 146

2.5.2.3 Embedded Methods 154

2.5.2.4 Feature Importance Using Graphics Processing Units (GPUs) 167

2.5.2.5 Feature Selection Using HephAIstos 168

Further Reading 170

3 Machine Learning Algorithms 175

3.1 Linear Regression 176

3.1.1 The Math 176

3.1.2 Gradient Descent to Optimize the Cost Function 177