CS516: Parallelization of Programs
Introduction GPUs and CUDA Programming
Vishwesh Jatala
Assistant Professor
Department of CSE
Indian Institute of Technology Bhilai
[email protected]
2023-24 W
1
Course Outline
■ Introduction
■ Overview of Parallel Architectures
■ Performance
■ Parallel Programming
❑ GPUs and CUDA programming
❑ CUDA thread organization
❑ Instruction execution
❑ GPU memories
❑ Synchronization
❑ Unified memory
■ Case studies
■ Extracting Parallelism from Sequential Programs
Automatically
2
Outline
■ GPUs and CUDA Programming Demos
3
Motivation
■ For many decades, the single core processors
were popular
❑ Instruction-level parallelism
❑ Core clock frequency
❑ Moore’s law
■ Mid-to late-1990s - power wall
❑ Power constraints
❑ Heat dissipation
■ Multicore processors, accelerators, such as
GPUs.
4
Why GPUs?
■ Multicore processors
❑ Task level parallelism
❑ Graphics rendering is
computationally
expensive
❑ Not efficient for
graphics applications
Images Source: Internet 5
Graphics Processing Units
■ The early GPU designs
❑ Specialized for graphics
processing only
❑ Exhibit SIMD execution
❑ Less programmable
NVIDIA GeForce 256
■ In 2007, fully
programmable GPUs
❑ CUDA released
Images Source: Internet 6
GPU Architecture
7
GPU Architecture
8
Parallelizing Programs on GPUs
9
Programming Models
■ CUDA (Compute Unified Device Architecture)
❑ Supports NVIDIA GPUs
❑ Extension of C programming language
❑ Popular in academia
■ OpenCL (Open Computing Language)
❑ Open source
❑ Supports various GPU devices
10
Introduction to CUDA Programming
GPU (Device)
(2) Kernel SM SM SM
Device Memory
(1) CPU to GPU (3) GPU to CPU
Data transfer Data transfer
Memory
CPU (Host)
11
Hello World
#include <stdio.h>
int main() {
printf("Hello World.\n"); Compile: gcc hello.c
Run: ./a.out
return 0; Hello World.
}
12
Hello World in GPU
#include <stdio.h>
#include <cuda.h>
__global__ void dkernel() {
printf(“Hello World.\n”); Compile: nvcc hello.cu
Run: ./a.out
}
Hello World.
int main() {
dkernel<<<1, 1>>>();
cudaDeviceSynchronize();
return 0;
}
13
Hello World in GPU
#include <stdio.h>
#include <cuda.h>
__global__ void dkernel() {
printf(“Hello World.\n”); Compile: nvcc hello.cu
} Run: ./a.out
No output
int main() {
dkernel<<<1, 1>>>();
return 0;
GPU Kernel launch is asynchronous!
}
14
Hello World in GPU
#include <stdio.h>
#include <cuda.h>
__global__ void dkernel() {
printf(“Hello World.\n”); Compile: nvcc hello.cu
Run: ./a.out
}
Hello World.
int main() {
dkernel<<<1, 1>>>();
cudaDeviceSynchronize();
return 0;
}
15
Hello World in Parallel in GPU
#include <stdio.h>
#include <cuda.h>
__global__ void dkernel() {
printf(“Hello World.\n”); Compile: nvcc hello.cu
} Run: ./a.out
Hello World.
int main() { Hello World.
32 times
dkernel<<<1, 32>>>(); …………….
Hello World.
cudaDeviceSynchronize();
return 0;
}
16
Example-1
#include <stdio.h>
#define N 100
int main() {
int i;
for (i = 0; i < N; ++i)
printf("%d\n", i * i);
return 0;
}
17
Example-1
#include <stdio.h>
#include <stdio.h> #include <cuda.h>
#define N 100 #define N 100
int main() { __global__ void fun() {
int i; printf("%d\n", threadIdx.x*threadIdx.x);
for (i = 0; i < N; ++i) }
printf("%d\n", i * i); int main() {
return 0; fun<<<1, N>>>();
} cudaDeviceSynchronize();
return 0;
}
18
GPU Hello World with a Global
19
Separate Memories
DRAM DRAM
PCI Express
Bus
CPU GPU
■ CPU and its associated (discrete) GPUs have separate
physical memory (RAM).
■ A variable in CPU memory cannot be accessed directly in
a GPU kernel.
■ A programmer needs to maintain copies of variables.
■ It is programmer's responsibility to keep them in sync.
20
CUDA Programs with Data Transfers
GPU (Device)
(2) Kernel SM SM SM
Device Memory
(1) CPU to GPU (3) GPU to CPU
Data transfer Data transfer
Memory
CPU (Host)
21
Data Transfer
■ Copy data from CPU to GPU
cudaMemcpy(gpulocation, cpulocation, size,
cudaMemcpyHostToDevice);
■ Copy data from CPU to GPU
cudaMemcpy(cpulocation, ppulocation, size,
cudaMemcpyDeviceToHost);
This means we need two copies of the same variable –
one on CPU another on GPU.
e.g., int *cpuarr, *gpuarr;
22
CPU-GPU Communication
#include <stdio.h>
#include <cuda.h>
__global__ void dkernel(char *arr, int arrlen) {
unsigned id = threadIdx.x;
if (id < arrlen) {
++arr[id];
}
}
int main() {
char cpuarr[] = "CS516", *gpuarr;
cudaMalloc(&gpuarr, sizeof(char) * (1 + strlen(cpuarr)));
cudaMemcpy(gpuarr, cpuarr, sizeof(char) * (1 + strlen(cpuarr)), cudaMemcpyHostToDevice);
dkernel<<<1, 32>>>(gpuarr, strlen(cpuarr));
cudaDeviceSynchronize(); // unnecessary.
cudaMemcpy(cpuarr, gpuarr, sizeof(char) * (1 + strlen(cpuarr)), cudaMemcpyDeviceToHost);
printf(cpuarr);
return 0;
}
23
Example
#include <stdio.h>
#include <stdio.h> #include <cuda.h>
#define N 100 #define N 100
__global__ void fun(int *a) {
int main() { a[threadIdx.x] = threadIdx.x * threadIdx.x;
}
int a[N], i;
int main() {
int a[N], *da;
for (i = 0; i < N; ++i)
int i;
a[i] = i * i;
cudaMalloc(&da, N * sizeof(int));
return 0; fun<<<1, N>>>(da);
cudaMemcpy(a, da, N * sizeof(int),
}
cudaMemcpyDeviceToHost);
Takeaway for (i = 0; i < N; ++i)
printf("%d\n", a[i]);
return 0;
}
24
References
■ CS6023 GPU Programming
❑ https://www.cse.iitm.ac.in/~rupesh/teaching/gpu/jan
20/
■ Miscellaneous resources from internet
■ https://developer.nvidia.com/blog/cuda-refresh
er-cuda-programming-model/
25
