Preface Acknowledgements
CHAPTER1 Introduction1
11 Heterogeneous Parallel Computing2
12 Architecture of a Modern GPU6
13 Why More Speed or Parallelism?8
14 Speeding Up Real Applications10
15 Challenges in Parallel Programming 12
16 Parallel Programming Languages and Models12
17 Overarching Goals14
18 Organization of the Book15
References 18
CHAPTER2 Data Parallel Computing19
21 Data Parallelism20
22 CUDA C Program Structure22
23 A Vector Addition Kernel 25
24 Device Global Memory and Data Transfer27
25 Kernel Functions and Threading32
26 Kernel Launch37
27 Summary38
Function Declarations38
Kernel Launch38
Built-in (Predefined) Variables 39
Run-time API39
28 Exercises39
References 41
CHAPTER3 Scalable Parallel Execution43
31 CUDA Thread Organization43
32 Mapping Threads to Multidimensional Data47
33 Image Blur: A More Complex Kernel 54
34 Synchronization and Transparent Scalability 58
35 Resource Assignment60
36 Querying Device Properties61
37 Thread Scheduling and Latency Tolerance64
38 Summary67
39 Exercises67
CHAPTER4 Memory and Data Locality 71
41 Importance of Memory Access Efficiency72
42 Matrix Multiplication73
43 CUDA Memory Types77
44 Tiling for Reduced Memory Traffic84
45 A Tiled Matrix Multiplication Kernel90
46 Boundary Checks94
47 Memory as a Limiting Factor to Parallelism97
48 Summary99
49 Exercises