CUDA

• nvidia developer
  • cuda-toolkit
  • gpu compute capability
• docs
  • quick start
  • support compiler
  • best practices guide
  • cuda c++ programming guide
• source
  • samples
  • github: nvidia/cuda-samples
• repos
  • cutlass

---

GPU Compute Capability

• gpu compute capability

nvidia-smi --query-gpu=compute_cap --format=csv

compute_cap
8.6

---

Code

Samples

git clone https://github.com/NVIDIA/cuda-samples.git

c++11_cuda

• Introduction: c++11_cuda

cd Samples/0_Introduction/c++11_cuda

Compile

make HOST_COMPILER=clang++ SMS="86" dbg=1
make HOST_COMPILER=g++ SMS="86" dbg=1
make HOST_COMPILER=g++-13 SMS="86" dbg=1

Run

./c++11_cuda

GPU Device 0: "Ampere" with compute capability 8.6

Read 3223503 byte corpus from ./warandpeace.txt
counted 107310 instances of 'x', 'y', 'z', or 'w' in "./warandpeace.txt"

---

Docs

• install
• clang: format
• api: driver, runtime
• huggingface
  • text embeddings inference
• docker
• nvidia
  • triton
  • libnvidia-container
  • dynamo
  • tensorRT, src/tensorrt
• leetgpu

---

Code

• Hello CUDA: hello_cuda, hello_cuda with C++
• Thread: thread_layout
• Device: device_query
• Vector: vector_add
• Matrix
  • add: matrix_add, matrix_add_large
  • mul: matrix_mul, matrix_mul_shared_memory, matrix_mul_shared_memory_large
• TensorRT: tensorrt
• Sync: sync, streams + event

---

Ref

• CUDA Books archive
• book: Programming Massively Parallel Processors
• book: CUDA Programming
• book: The Art of HPC
• youtube: CUDA Programming Course – High-Performance Computing with GPUs
• youtube: GPU MODE
• GPU Glossary
• UIUC: Introduction to Parallel Programming with CUDA