HPC notes
next week builkding 550 d school
shesha Kirshnapura: Intel (week 6 visit ), looking for interns and reseachers for
thier labs
Make sure you name your files correclty for asignemnts.
Lec 2
Open HPC:
Free for maanging high performmance clusters. open source.
IBM ,etc charge a lot for similar softwares.. but HOC is free
Rocky Linux
OPENHPC file system:
Linux filesystem: / --> root
/etc, /usr --> /lib64, /bin, /home --> /student , /test (users), /opt
Open HPC uses chroot for building compute node VNFS i mages
Allows maanigng the compute node in a virtualized env. like a standalone
worksattion or 'conatiner'.. like an OS but no access to hardware (host OS has
hardware acces bit this 'sub os' doesnot..
chroot is the root for the sub directory or the container. NOT the root of main OS.
VNFS image uses 'warewulf database'
Apptainer container on HPC clusters
https://github.com/stanfordhpccenter/apptainer_lecture
- mounts a new file system somehwere nin your host filesystem tree
- squash
- nosuid flag on new file systems . cant use any provildeges for security reasons.
this can turn into a problem
- Linux Namespaces: (Namespace takes a srt of global resources and divide it into
ssemfgnts for difernt processes.)
- Apptainer utilizes user namepsace for unprivildged processes to mount
-
- a self-contained execution environment platform that enable flexibility and
portability of your MPI appliction.
- Problems that arise wiht mpi. If the applcation is on anoter node in a contaier,
and the applcaition is in another conatiner, then mpi cant find it since app is ina
conatiners and it is in a different root directrory.
- SO we do mpiexex < > apptainer exec ....
- apptainer made using gpus easy on containers
- the problem wth using GPU in congainer is
- -- nv in (if you in nvidia host) or --rocm (if you are in ad host) to copy the
driver files from the host system to the container.
-
- mpi wire up prbems( fuzzball sovles it)
MIcroprocessro Archietectures:
scale up vs scale out:
scale up: add nodes in a single computer in a single chasis so it looks like on
server comp
scale out: build a network of any server comps
Blocking vs non blocking switch:
non blocking switch: when every node is conected to every ohter node
blocking: when it isnt
4 types of super comps
simd, misd etc etc