Programming Your GPU with OpenMP - Timothy G. Mattson

. .

Résumé :
OpenMP is a widely used language for programming the nodes in a parallel computer. Those nodes are now heterogeneous, including a GPU alongside the traditional CPU--...

Biographie:
s Hierarchical Parallelism 127
6.1 Parallel Threads 128
6.2 League of Teams of Threads 130
6.2.1 Controlling the Number of Teams and Threads 132
6.2.2 Distributing Work between Teams 135
6.3 Hierarchical Parallelism in Practice 139
6.3.1 Example: Batched Matrix Multiplication 140
6.3.2 Example: Batched Gaussian Elimination 142
6.4 Hierarchical Parallelism and the Loop Directive 143
6.4.1 Combined Constructs that Include Loop 145
6.4.2 Reductions and Combined Constructs 146
6.4.3 The Bind Clause 146
6.5 Summary 149
7 Revisiting Data Movement 151
7.1 Manipulating the Device Data Environment 151
7.1.1 Allocating and Deleting Variables 155
7.1.2 Map Type Modifiers 158
7.1.3 Changing the Default Mapping 160
7.2 Compiling External Functions and Static Variables for the Device 162
7.3 User-Defined Mappers 168
7.4 Team-Only Memory 173
7.5 Becoming a Cartographer: Mapping Device Memory by Hand 179
7.6 Unified Shared Memory for Productivity 185
7.7 Summary 189
8 Asynchronous Offload to Multiple GPUs 191
8.1 Device Discovery 193
8.2 Selecting a Default Device 194
8.3 Offload to Multiple Devices 196
8.3.1 Reverse Offload 198
8.4 Conditional Offload 200
8.5 Asynchronous Offload 201
8.5.1 Task Dependencies 202
8.5.2 Asynchronous Data Transfers 206
8.5.3 Task Reductions 208
8.6 Summary 210
9 Working with External Runtime Environments 213
9.1 Calling External Library Routines from OpenMP 213
9.2 Sharing OpenMP Data with Foreign Functions 217
9.2.1 The Need for Synchronization 221
9.2.2 Example: Sharing OpenMP Data with cuBLAS 222
9.3 Using Data from a Foreign Runtime with OpenMP 223
9.3.1 Example: Sharing cuBLAS Data with OpenMP 227
9.3.2 Avoiding Unportable Code 229
9.4 Direct Control of Foreign Runtimes 231
9.4.1 Query Properties of the Foreign Runtime 234
9.4.2 Using the Interop Construct to Correctly
Synchronize with Foreign Functions 238
9.4.3 Non-blocking Synchronization with a Foreign Runtime 242
9.4.4 Example: Calling CUDA Kernels without Blocking 245
9.5 Enhanced Portability Using Variant Directives 248
9.5.1 Declaring Function Variants 250
9.5.1.1 OpenMP Context and the Match Clause 253
9.5.1.2 Modifying Variant Function Arguments 255
9.5.2 Controlling Variant Substitution with the Dispatch Construct 257
9.5.3 Putting It All Together 259
10 OpenMP and the Future of Heterogeneous Computing 263
Appendix: Reference Guide 265
A.1 Programming a CPU with OpenMP 266
A.2 Directives and Constructs for the GPU 268
A.2.1 Parallelism with Loop, Teams, and Worksharing Constructs 272
A.2.2 Constructs for Interoperability 275
A.2.3 Constructs for Device Data Environment Manipulation 278
A.3 Combined Constructs 281
A.4 Internal Control Variables, Environment Variables, and OpenMP API Functions 283
Glossary 287
References 301
Subject Index 305...

Sommaire:
Tom Deakin and Timothy G. Mattson...