Machine Learning and Big Data with Kdb+/Q - Aris Galiotos

. .

Résumé :
..

Biographie:

JAN NOVOTNY is an eFX quant trader at Deutsche Bank. Previously, he worked at the Centre for Econometric Analysis on high-frequency econometric models. He holds a PhD from CERGE-EI, Charles University, Prague.

PAUL A. BILOKON is CEO and founder of Thalesians Ltd and an expert in algorithmic trading. He previously worked at Nomura, Lehman Brothers, and Morgan Stanley. Paul was educated at Christ Church College, Oxford, and Imperial College.

ARIS GALIOTOS is the global technical lead for the eFX kdb+ team at HSBC, where he helps develop a big data installation processing billions of real-time records per day. Aris holds an MSc in Financial Mathematics with Distinction from the University of Edinburgh.

FR?D?RIC D?L?ZE is an independent algorithm trader and consultant. He has designed automated trading strategies for hedge funds and developed quantitative risk models for investment banks. He holds a PhD in Finance from Hanken School of Economics, Helsinki....

Sommaire:

Preface xvii

About the Authors xxiii

Part One Language Fundamentals

Chapter 1 Fundamentals of the q Programming Language 3

1.1 The (Not So Very) First Steps in q 3

1.2 Atoms and Lists 5

1.3 Basic Language Constructs 14

1.4 Basic Operators 19

1.5 Difference between Strings and Symbols 31

1.6 Matrices and Basic Linear Algebra in q 33

1.7 Launching the Session: Additional Options 35

1.8 Summary and How-To's 38

Chapter 2 Dictionaries and Tables: The q Fundamentals 41

2.1 Dictionary 41

2.2 Table 44

2.3 The Truth about Tables 48

2.4 Keyed Tables are Dictionaries 50

2.5 From a Vector Language to an Algebraic Language 51

Chapter 3 Functions 57

3.1 Namespace 59

3.2 The Six Adverbs 60

3.3 Apply 72

3.4 Protected Evaluations 75

3.5 Vector Operations 76

3.6 Convention for User-Defined Functions 79

Chapter 4 Editors and Other Tools 81

4.1 Console 81

4.2 Jupyter Notebook 82

4.3 GUIs 84

4.4 IDEs: IntelliJ IDEA 90

4.5 Conclusion 92

Chapter 5 Debugging q Code 93

5.1 Introduction to Making It Wrong: Errors 93

5.2 Debugging the Code 100

5.3 Debugging Server-Side 102

Part Two Data Operations

Chapter 6 Splayed and Partitioned Tables 107

6.1 Introduction 107

6.2 Saving a Table as a Single Binary File 108

6.3 Splayed Tables 110

6.4 Partitioned Tables 113

6.5 Conclusion 119

Chapter 7 Joins 121

7.1 Comma Operator 121

7.2 Join Functions 125

7.3 Advanced Example: Running TWAP 144

Chapter 8 Parallelisation 151

8.1 Parallel Vector Operations 152

8.2 Parallelisation over Processes 155

8.3 Map-Reduce 155

8.4 Advanced Topic: Parallel File/Directory Access 158

Chapter 9 Data Cleaning and Filtering 161

9.1 Predicate Filtering 161

9.2 Data Cleaning, Normalising and APIs 163

Chapter 10 Parse Trees 165

10.1 Definition 166

10.2 Functional Queries 171

Chapter 11 A Few Use Cases 181

11.1 Rolling VWAP 181

11.2 Weighted Mid for N Levels of an Order Book 183

11.3 Consecutive Runs of a Rule 185

11.4 Real-Time Signals and Alerts 186

Part Three Data Science

Chapter 12 Basic Overview of Statistics 191

12.1 Histogram 191

12.2 First Moments 196

12.3 Hypothesis Testing 198

Chapter 13 Linear Regression 229

13.1 Linear Regression 230

13.2 Ordinary Least Squares 231

13.3 The Geometric Representation of Linear Regression 233

13.4 Implementation of the OLS 240

13.5 Significance of Parameters 243

13.6 How Good is the Fit: R2 244

13.7 Relationship with Maximum Likelihood Estimation and AIC with Small Sample Correction 248

13.8 Estimation Suite 252

13.9 Comparing Two Nested Models: Towards a Stopping Rule 254

13.10 In-/Out-of-Sample Operations 257

13.11 Cross-validation 262

13.12 Conclusion 264

Chapter 14 Time Series Econometrics 265

14.1 Autoregressive and Moving Average Processes 265

14.2 Stationarity and Granger Causality 285

14.3 Vector Autoregression 287

Chapter 15 Fourier Transform 301

15.1 Complex Numbers 301

15.2 Discrete Fourier Transform 308

15.3 Addendum: Quaternions 314

15.4 Addendum: Fractals 321

Chapter 16 Eigensystem and PCA 325

16.1 Theory 325

16.2 Algorithms 327

16.3 Implementation of Eigensystem Calculatio...