High Performance Scientific computing
Lecture 4
S. Gopalakrishnan!
Memory Issues
Memory hierarchy
Faster
Costlier
Typical Hierarchy
Memory Latency Problem
Cache/MM virtual memory
Processor-DRAM Memory Performance Gap
Motivation for Memory Hierarchy
C µProc
1000CPU 8B a 32 B Memory 4 KB
CPU Memory disk
60%/yr.
disk
regs c (2X/1.5yr)
Performance
regs
h
100 Processor-Memory
e
Performance Gap:
! Notice 10
that the data width is changing (grows 50% / year)
• Why? DRAM
! Bandwidth: Transfer rate between various levels 5%/yr.
1 (2X/15 yrs)
• CPU-Cache: 24 GBps
1980
1984
1986
1988
1989
1990
1992
1994
1996
1998
1999
1981
1982
1983
1985
1987
1991
1993
1995
1997
2000
• Cache-Main: 0.5-6.4GBps
• Main-Disk: 187MBps (serial ATA/1500)
Time
ECE232: Memory Hierarchy 5 Adapted from Computer Organization and Design,Patterson&Hennessy, UCB, Kundu,UMass Koren
Source:Ece
ECE232: Memory Hierarchy 12 232 Umass-Amherst
Adapted from Computer Organization and Design,Patterson&Hennessy, UCB, Kundu,UMass Koren
Virtual Memory and Paging
Virtual memory Physical
(per process) memory
Another
process's
memory
RAM
Source: www. wikipedia.com
Disk
Memory Hierarchy Terminology
Memory Hierarchy: Terminology
! Hit: data appears in upper level in block X
! Hit Rate: the fraction of memory accesses found in the upper
level
! Miss: data needs to be retrieved from a block in the lower
level (Block Y)
! Miss Rate = 1 - (Hit Rate)
! Hit Time: Time to access the upper level which consists of
Time to determine hit/miss + upper level access time
! Miss Penalty: Time to replace a block in the upper level +
Time to deliver the block to the processor
! Note: Hit Time << Miss Penalty Lower Level
To Processor Upper Level
Block Y
From Processor
Block X
Source: ECE 232 Umass-Amherst
ECE232: Memory Hierarchy 15 Adapted from Computer Organization and Design,Patterson&Hennessy, UCB, Kundu,UMass Koren
Current Memory Hierarchy
Current Memory Hierarchy
Memory Latency Problem
Processor
Processor-DRAM Memory Performance Gap
Motivation for Memory Hierarchy
Control µProc
Secondary
1000
Main 60%/yr.
Memory
(2X/1.5yr)
Performance
L2 Memory
Data-
regs
100 L1 Cache Processor-Memory
path Cache
Performance Gap:
10 (grows 50% / year)
DRAM
Speed(ns): 1ns 2ns 6ns 100ns 10,000,000ns
5%/yr.
Size (MB): 1 0.0005 0.1 1-4 1000-6000 500,000
(2X/15 yrs)
Cost ($/MB): -- $10 $3 $0.01 $0.002
1980
1984
1986
1988
1989
1990
1992
1994
1996
1998
1999
1981
1982
1983
1985
1987
1991
1993
1995
1997
2000
Technology: Regs SRAM SRAM DRAM Disk
• Cache - Main memory: Time
Speed
• Main
ECE232: memory
Memory Hierarchy 5 – Disk (virtual memory): Capacity
Adapted from Computer Organization and Design,Patterson&Hennessy, UCB, Kundu,UMass Koren
Source:Ece
ECE232: Memory Hierarchy 16 232 Umass-Amherst
Adapted from Computer Organization and Design,Patterson&Hennessy, UCB, Kundu,UMass Koren
Introduction to Parallel Programming
Shared'Memory,Processing,
,Each,processor,can,access,the,en6re,data,space,
,
– Pro’s,
• Easier,to,program,
• Amenable,to,automa6c,parallelism,
• Can,be,used,to,run,large,memory,serial,programs,
– Con’s,
• Expensive,
• Difficult,to,implement,on,the,hardware,level,
• Processor,count,limited,by,conten6on/coherency,(currently,around,512),
• Watch,out,for,“NU”,part,of,“NUMA”,
Distributed*–*Memory*Machines*
! Each*node*in*the*computer*has*a*locally*addressable*memory*space*
! The*computers*are*connected*together*via*some*high:speed*network*
– Infiniband,*Myrinet,*Giganet,*etc..*
• Pros*
– Really*large*machines*
– Size*limited*only*by*gross*physical*
consideraFons:*
• Room*size*
• Cable*lengths*(10’s*of*meters)*
• Power/cooling*capacity*
• Money!*
– Cheaper*to*build*and*run*
• Cons*
– Harder*to*program*
* *Data*Locality*
MPPs$(Massively$Parallel$Processors)$
Distributed$memory$at$largest$scale.$$OTen$shared$memory$
$at$lower$hierarchies.$
• IBM$BlueGene/L$(LLNL)$
– 131,072$700$Mhz$processors$
– 256$MB$or$RAM$per$processor$
– Balanced$compute$speed$with$interconnect$
! Red$Storm$(Sandia$NaJonal$Labs)$
– 12,960$Dual$Core$2.4$Ghz$Opterons$
– 4$GB$of$RAM$per$processor$
– Proprietary$SeaStar$interconnect$
fundamentally different design
Comparison of CPU vs GPU Architecture
philosophies.
ALU ALU
Control
ALU ALU
CPU GPU
Cache
DRAM DRAM
Source: Prof. Wen-mei W. Hwu UIUC
©Wen-mei W. Hwu and David Kirk/NVIDIA, Chile,
G2S3
Parallelization
GPU vs CPU computingGPU CPU Analogy
It is more effective to deliver Pizza’s through light duty scooters
rather than big truck. Similarly effective to use several lightweight
GPU processors for parallel tasks.
GPU Performance
Performance Advantage of GPUs
Peak performance increase
• An enlarging peak performance
Calculation advantage:
~ 1 TFlop on Desktop
– Calculation:
Memory1 TFLOPS vs. 100~GFLOPS
Bandwidth 150 GB/s
– Memory Bandwidth: 100-150 GB/s vs. 32-64 GB/s
Courtesy: John Owens
– GPU in every PC and workstation – massive volume and potential
source: top500.org
source: top500.org
Compute Unified Device Architecture
(CUDA)
• CUDA set of APIs (application program interface)
to use GPU’s for general purpose computing
• Developed and released by NVIDIA Inc. Works
only on NVIDIA GPU hardware
• Works on commercial GPU’s and as well as
specialized ones for scientific computing (Tesla)
• CUDA compiler supports C programming
language. Extensions to FORTRAN are possible.
• Opensource alternative is OpenCL.
