Starflood is an open-source astrophysical simulation code written in C.

• Cosmological N-body simulation
  • Gravity solver using pairwise summation (aka particle-particle method), O(N²)
• Parallelization using OpenMP (compiler directive-based)
  • Device/GPU compute when using a toolchain with offloading support
• Visualization
  • Accumulation rasterization using atomic operations
  • Spatial anti-aliasing (Gaussian window function stochastic sampling)
• File I/O
  • Timesteps from the simulation can be saved in a raw binary format.
  • Visualization can be done during a run, or later after it has been completed.
    • The frame sequence is saved as a series of individual images, which can be encoded using tools such as ffmpeg.
    • PFM graphic image file format (.pfm)

• Add physically accurate initial conditions
  • Andromeda-Milky Way collision
  • Molecular cloud/Stellar nursery
• Add support for distributed computing
  • Message Passing Interface (MPI)
• Add support for volume rendering to visualization
• Improve physical solver(s)
  • Add support for Barnes-Hut tree method
  • Add support for Fast Multipole Method (FMM)

• Create Your Own Smoothed-Particle Hydrodynamics Simulation (With Python) by Philip Mocz
  • Used as a reference implementation of SPH, which is nice since the Wikipedia article only contains mathematical formulae/equations!
• Dynamics and Astrophysics of Galaxies by Jo Bovy
  • Web-based book
• GADGET-4 (GAlaxies with Dark matter and Gas intEracT) by Volker Springel et al.
  • Used as a reference for what a well-organized astrophysical simulation codebase should look like. Also an awesome collaboration/project that was an inspiration to me.
• The Barnes-Hut Approximation: Efficient computation of N-body forces by Jeffrey Heer
  • An epic article with interactive visualizations outlining the Barnes-Hut algorithm.
• STARFORGE: Star Formation in Gaseous Environments by Mike Grudić et al.
  • Another cool collaboration/project that inspires me.
• PCG, A Family of Better Random Number Generators by Melissa E. O'Neill
  • Fast/high-quality/simple hash functions used for generating the simulation intial conditions and graphical visualizations.
  • Hash Functions for GPU Rendering by Mark Jarzynski and Marc Olano
    • Source of the pcg4d() variant of PCG.
• stb: single-file public domain (or MIT licensed) libraries for C/C++
  • Source of the stb_image.h and stb_image_write.h C headers for image file I/O. I highly recommend anybody working on a small C/C++ project like this take a look at the available stb headers.

First, clone the Starflood repository:

git clone https://github.com/Zi7ar21/starflood.git

Then, change to the repository root:

cd starflood

There are a few parameters at the top of src/config.h worth looking at, such as NUM_BODIES:

// number of bodies in the simulation (N)
#define NUM_BODIES 65536

First, create a directory for the build files.

mkdir -p build

First, make any desired changes to the Makefile. Some lines are commented/uncommented near the top of the file that you might want to tweak.

To compile in parallel, use -j to specify the number of jobs:

make -j$(nproc) all

To clean up after the build (required if any modificaitons have been made to the source code):

make clean

• -ffast-math: Allows replacement of standard math library functions with native instructions (i.e. sqrt() becomes the native x86 SSE instruction sqrtss). Note: This flag may cause runs to be non-deterministic across compilers and vendors!
• -march=native: Generate binaries optimized for the compiler host machine (i.e. cache optimizations and enables any supported SIMD instruction sets, such as x86 AVX or ARM NEON).

Tip: On Arch Linux, the NVIDIA HPC SDK has a package (extra/nvhpc).

For OpenMP offloading using the NVIDIA HPC Compilers:

nvc -gpu=ccnative -mp=gpu -fopenmp -g -march=native -O4 -pedantic -std=c99 -Wall -Wextra -Wshadow -o build/starflood src/*.c -lm

• -mp=gpu: Enable compilation of OpenMP target directives for the GPU.
• -gpu=ccnative: Generates codes only for visible GPUs on the compiler host machine.

Disk I/O can be annoying during development if you aren't planning on keeping run data or large frame sqeuences around for a while.

Thankfully, Linux has an easy way to create a virtual memory filesystem (sometimes called a "ramdisk" by Windows users) using the mount comand:

mkdir -p out

sudo mount -o size=4G -t tmpfs tmpfs out

Where size=4G indicates a filesystem size of 4 GiB.

Remember to unmount the tmpfs when you are finished!

sudo umount out

Please see Tmpfs in The Linux Kernel documentation or tmpfs man page for more details.

man tmpfs

This section is still a work-in-progress!

mkdir -p out/sim out/vis

nice is a tool that can be used to run a program with a higher/lower niceness value. The following command will run ./build/starflood with a niceness value 9 higher than the shell nice was called from:

nice -9 ./build/starflood

A higher niceness means the scheduler will deprioritize the process, meaning any other programs on your system will keep running smoothly.

ffmpeg -y -r 30 -framerate 30 -i ./out/vis/step_%04d.pfm -c:v h264_nvenc -vf "scale=in_transfer=linear:out_transfer=bt709" -b:v 8M -pix_fmt yuv420p -an -sn -dn -g 15 -bf 2 -preset slow -tune hq -profile:v main -rc-lookahead 16383 -spatial_aq 1 -temporal_aq 1 -coder cabac -b_ref_mode middle starflood_out.mp4 && mpv --loop starflood_out.mp4

A simple way to time execution is using GNU Time:

/usr/bin/time -v ./build/starflood

You can collect and measurements from Hardware performance counters during execution (elevated privileges required for perf stat):

sudo perf stat -ddd sudo -u $USER ./build/starflood

[to-do]

Compilers such as Clang/GCC support profile-guided optimization (PGO), a technique where profiling data collected during execution can be used to further optimize generated code for the target machine.

Useful article: https://ddmler.github.io/compiler/2018/06/29/profile-guided-optimization.html

See "Profile Guided Optimization" in the Clang Compiler User's Manual.

See "3.12 Options That Control Optimization" in the GNU Manual.