Responsible Data Science - Grant Fleming

. .

Résumé :

Introduction xix

Part I Motivation for Ethical Data Science and Background Knowledge 1

Chapter 1 Responsible Data Science 3

The Optum Disaster 4

Jekyll and Hyde 5

Eugenics 7

Galton, Pearson, and Fisher 7

Ties between Eugenics and Statistics 7

Ethical Problems in Data Science Today 9

Predictive Models 10

From Explaining to Predicting 10

Predictive Modeling 11

Setting the Stage for Ethical Issues to Arise 12

Classic Statistical Models 12

Black-Box Methods 14

Important Concepts in Predictive Modeling 19

Feature Selection 19

Model-Centric vs. Data-Centric Models 20

Holdout Sample and Cross-Validation 20

Overfitting 21

Unsupervised Learning 22

The Ethical Challenge of Black Boxes 23

Two Opposing Forces 24

Pressure for More Powerful AI 24

Public Resistance and Anxiety 24

Summary 25

Chapter 2 Background: Modeling and the Black-Box Algorithm 27

Assessing Model Performance 27

Predicting Class Membership 28

The Rare Class Problem 28

Lift and Gains 28

Area Under the Curve 29

AUC vs. Lift (Gains) 31

Predicting Numeric Values 32

Goodness-of-Fit 32

Holdout Sets and Cross-Validation 33

Optimization and Loss Functions 34

Intrinsically Interpretable Models vs. Black-Box Models 35

Ethical Challenges with Interpretable Models 38

Black-Box Models 39

Ensembles 39

Nearest Neighbors 41

Clustering 41

Association Rules 42

Collaborative Filters 42

Artificial Neural Nets and Deep Neural Nets 43

Problems with Black-Box Predictive Models 45

Problems with Unsupervised Algorithms 47

Summary 48

Chapter 3 The Ways AI Goes Wrong, and the Legal Implications 49

AI and Intentional Consequences by Design 50

Deepfakes 50

Supporting State Surveillance and Suppression 51

Behavioral Manipulation 52

Automated Testing to Fine-Tune Targeting 53

AI and Unintended Consequences 55

Healthcare 56

Finance 57

Law Enforcement 58

Technology 60

The Legal and Regulatory Landscape around AI 61

Ignorance Is No Defense: AI in the Context of Existing Law and Policy 63

A Finger in the Dam: Data Rights, Data Privacy, and Consumer Protection Regulations 64

Trends in Emerging Law and Policy Related to AI 66

Summary 69

Part II The Ethical Data Science Process 71

Chapter 4 The Responsible Data Science Framework 73

Why We Keep Building Harmful AI 74

Misguided Need for Cutting-Edge Models 74

Excessive Focus on Predictive Performance 74

Ease of Access and the Curse of Simplicity 76

The Common Cause 76

The Face Thieves 78

An Anatomy of Modeling Harms 79

The World: Context Matters for Modeling 80

The Data: Representation Is Everything 83

The Model: Garbage In, Danger Out 85

Model Interpretability: Human Understanding for Superhuman Models 86

Efforts Toward a More Responsible Data Science 89

Principles Are the Focus 90

Nonmaleficence 90

Fairness 90

Transparency 91

Accountability 91

Privacy 92

Bridging the Gap Between Principles and Practice with the Responsible Data Science (RDS) Framework 92

Justification 94

Compilation 94

Preparation 95

Modeling 96

Auditing 96

Summary 97

Chapter 5 Model Interpretability: The What and the Why 99

The Sexist R?sum? Screener 99

The Necessity of Model Interpretability 101

Sommaire:

GRANT FLEMING is a Data Scientist at Elder Research Inc. His professional focus is on machine learning for social science applications, model interpretability, civic technology, and building software tools for reproducible data science.

PETER BRUCE is the Senior Learning Officer at Elder Research, Inc., author of several best-selling texts on data science, and Founder of the Institute for Statistics Education at Statistics.com, an Elder Research Company.