This new Vlasiator release adds:

• Full support for GPU computation on HPC environments to Vlasiator, targeting both Nvidia (CUDA) and AMD (HIP) platforms.
• A subgrid-scale wave-particle scattering model (implemented as velocity space diffusion) has been added for magnetosheath waves.
• Vorticity and temperature / pressure anisotropy can now be taken into account for mesh refinement criteria.
• Improved the code's runtime resilience by including a mechanism to write frequent recovery files using a circular buffer scheme.
• The code is now ported (or portable) to all EuroHPC machines, as of December 2025.
• In addition, there are a large number of bugfixes to time-varying inflow, system boundary handling, ionosphere stability and overall quality of life.

More detailed list of what's Changed:

• Added blocks gather code for FALLBACK vectors by @markusbattarbee in #444
• Vec8Simple by @ykempf in #445
• Some openacc experiments in the acceleration solver. by @ursg in #446
• Support of MPI ranks sharing GPUs. by @ykempf in #447
• Restructured OpenACC acceleration solver to parallelize over columns. by @ursg in #448
• Cudasiator mutator, split plm/ppm/pqm etc back into .cuh files by @markusbattarbee in #595
• Cudasiator realf vectorclass by @markusbattarbee in #597
• Implementation of CUDA streams (one per thread) by @markusbattarbee in #616
• Cudasiator by @markusbattarbee in #790
• Vlasiator gpu by @markusbattarbee in #799
• Added testpackage script for mahti_cuda by @markusbattarbee in #803
• ARCH fixes to LUMI-G compilation, squelch some warnigns, added NOMAD … by @markusbattarbee in #805
• Cudasiator pre-NOMAD-Hackathon fixes by @markusbattarbee in #806
• Cudasiator preparation for the NOMAD hackathon by @markusbattarbee in #808
• Fix project initialization fixing due to bogus free by @ursg in #809
• Deactivated xnack on LUMI, added trial config for all datareducers by @markusbattarbee in #810
• Add -march=znver3 to lumi gpu makefiles by @ursg in #811
• Don't NACK in the lumi gpu testpackage script by @ursg in #813
• Recapacitate splitVectors in acceleration if they are too small. by @ursg in #812
• Added method for applying reservation and call it after LB. Also swit… by @markusbattarbee in #814
• Stop your damn timers! by @lkotipal in #815
• Fix timer by @lkotipal in #816
• Fix arch-interface issues in DRO and add HIP kernel launch error checking by @hokkanen in #817
• Fix kernel launch issue and dynamic shared memory type size issue by @hokkanen in #819
• Fix issue with restart_read testpackage test in GPU branch by @markusbattarbee in #835
• Gpu fix restarts, update to dev by @markusbattarbee in #837
• Prefer --offload-arch over the deprecated --amdgpu-target and fix commit parsing in Makefile. by @sfantao in #844
• Use multiple handlers of device mesh wrapper to avoid relocatable code device builds by @sfantao in #868
• Avoid rdc device code mk2 by @markusbattarbee in #872
• Vlasiator gpu fix cpu compilation and running by @markusbattarbee in #922
• Purge all remaining references to velocity mesh refinement by @markusbattarbee in #923
• Vlasiator gpu bring up to date with dev by @markusbattarbee in #924
• Vlasiator gpu fix timevarying by @markusbattarbee in #928
• fix gpu translation DZ access from pencils by @markusbattarbee in #929
• Vlasiator_gpu update on LUMI to 23.09 by @markusbattarbee in #936
• Fix compute-time loop by @hokkanen in #938
• Updated Mahti_cuda makefile to use gcc-10.4.0 and cuda-12.1.1 by @markusbattarbee in #937
• Reverting submodule urls to https instead of ssh, for easier accessib… by @markusbattarbee in #945
• Gpu update - new block adjustment algorithm and plenty of other improvements by @markusbattarbee in #958
• Gpu remove map add by @markusbattarbee in #962
• Fix GPU branch crash on several MPI Tasks by @markusbattarbee in #965
• Optimizations to translation memory ordering, and fix to broken ifdefs by @markusbattarbee in #972
• Vlasiator gpu hashinator update by @markusbattarbee in #979
• GPU: Update Hashinator interface to use templated prefetch deactivations by @markusbattarbee in #978
• Removed block offset evaluation kernel in ACC; included functionality… by @markusbattarbee in #980
• Add OMP support for ARCH host backend by @hokkanen in #969
• Vlasiator gpu branch, fix dynamic AMR operations by @markusbattarbee in #981
• Purged config references to the GPUBLOCKS / CUDABLOCKS parameter and … by @markusbattarbee in #983
• Gpu single trans kernel launch dev by @markusbattarbee in #1021
• Split cell transfer into a number of passes in load balance. by @ykempf in #1017
• Different trans thread guide by @markusbattarbee in #1032
• Update vlasiator to use updated fsgrid constructor by @cscjlan in #1019
• Lumi 2403 makefile by @JonasSuni in #1031
• GCC 13.2.0 carrington by @ykempf in #1035
• Update DCCRG commit by @markusbattarbee in #1037
• Caching of DCCRG <-> FSgrid coupling by @veetihaaja in #1012
• Better documentation for alpha1 constituent weighting by @lkotipal in #1001
• Fix check vlasiator cfg script by @ykempf in #1024
• Add jemalloc purge calls to after shrink_to_fit and deallocation of r… by @markusbattarbee in #1027
• Makefile (and small library build script fix) for Meluxina. by @ursg in #1041
• Use curl instead of wget in build_libraries.sh by @ursg in #1047
• Make chunk data script use 5 GB chunks, for Allas by @JonasSuni in #1046
• Correct EGradPe/derivatives communications in subcycling by @ykempf in #1038
• fsgrid RAII by @lkotipal in #918
• No L2 NOT_SYSBOUNDARY EXB nudge by @ykempf in #1050
• GradPe fix cleanup by @ykempf in #1051
• Build testpackage data with new gcc version on carrington. by @ursg in #1054
• Fixed typo in documentation. by @ykempf in #1056
• Reverse direction when stepping back to hit exact radius when tracing. by @ykempf in #1057
• New amr criteria by @lkotipal in #1022
• Multipop pressure anistropy by @lkotipal in #1052
• Ghost translation dev merge by @ursg in #1060
• Ghost translation by @markusbattarbee in #1006
• Dccrg RAII by @lkotipal in #953
• Multipop pressure anistropy by @lkotipal in #1061
• Update C++ standard from C++17 to C++20 in Makefile.lumi_gnu_2403 by @JonasSuni in #1063
• Fix a broken error message in pencil construction. by @ursg in #1071
• Slope limiter pass by reference by @markusbattarbee in #1069
• Limit tracing volume by @ykempf in #1055
• Simple matplotlib plotter to help setting VDF parameters. by @ykempf in #1073
• Inspection script mechanism to use srun overlap. Works on LUMI at least. by @ykempf in #1068
• Node hours report by @ArnaudLalague in #1053
• Use ARCH=arch in VLSV build by @ykempf in ...

This release brings various minor quality-of-life improvements and security updates.

• dynamic AMR criteria updated to match Kotipalo et al 2024: vg_amr_alpha -> vg_amr_alpha1, vg_amr_jperb -> vg_amr_alpha2, scaling fixes
• Ionospheric parameters written to vlsv files
• Added option to deactivate refinement by Cartesian coordinate thresholds
• Fixes to timers
• New testpackage and CI structure
• Cleaned up sample configuration files
• Fix to 3D dipole field in solar wind inflow cells

Auto-generated release notes follow:

What's Changed

• Use newer docker image in CI runs. by @ursg in #899
• Github actions workflow to automatically sync branches to HLRS gitlab by @ursg in #901
• static dipole copysphere.zeroPerB = 1 in testpackage Magnetopshere_small by @ykempf in #905
• Fix gitlab sync action yaml syntax by @ursg in #910
• Fix git remote syntax in gitlab sync action. by @ursg in #911
• vlsvExtract std containers dis-ambiguation by @alhom in #913
• refine_radius for dAMR version support, a QoL improvements: output b … by @alhom in #912
• updated dccrg and fsgrid submodule commits by @markusbattarbee in #907
• updated dccrg and fsgrid submodule commits by @markusbattarbee in #908
• Use Github "Retry Step" Action to wait for vlasiator binary on lustre. by @ursg in #921
• Mpi in block adjustment by @markusbattarbee in #919
• Refinement param renames by @lkotipal in #920
• Some explanations of config option groups in sample cfg by @ykempf in #917
• Output time difference in vlsvdiff by @lkotipal in #925
• Fix CI runs on RISCV failing, probably due to old libraries still lingering in the work directory. by @ursg in #931
• Makefile and github actions steps to build on apple M1 by @ursg in #930
• Use mpirun instead of srun for tool runs on carrington for CI by @ursg in #934
• Update appleM1 CI build boost version. by @ursg in #935
• Cross-compile on BSC RiscV cluster for much faster CI builds by @ursg in #941
• Squelch a big bunch of unused variable warnings. by @ursg in #942
• Reverting submodule urls to https instead of ssh, for easier accessib… by @alhom in #932
• Fix Warnings by @lkotipal in #900
• Face neighbor caching by @lkotipal in #933
• Refinement hotfix by @lkotipal in #914
• Amr translation cleanup by @markusbattarbee in #767
• Minor moments fixes by @markusbattarbee in #947
• Amr translation cleanup addon by @alhom in #948
• Bug fix (Add parameters into the VLSV file) by @ArnaudLalague in #950
• Add support for DCCRG/FSgrid split feature (requires support from DCCRG and FSgrid) by @hokkanen in #926
• Push to GitLab only from fmihpc by @alhom in #954
• Reworked testpackage in a major fashion. by @markusbattarbee in #927
• Added Eigen as a submodule by @alhom in #890
• Move spatial cell to cpu-specific file by @markusbattarbee in #955
• Concurrency check and trap to scancel testpackage_run if caught cancel signal by @alhom in #961
• Remove . /etc/profile from generate_version.sh by @astatide in #963
• Updates to the Kelvin-Helmholtz instability project by @vetarvus in #960
• Fix ig_rhon units incorrectly being a mass density by @ursg in #971
• Update github artefact actions to v4. by @ursg in #968
• Add Makefile for MareNostrum 5's general partition by @astatide in #964
• Ci carrington fs ops to srun by @ykempf in #974
• Fix crash if OMPI_MCA_io is not set in environment. by @ursg in #984
• Dev up-to-speed by @alhom in #967
• Don't checkout submodules in testpackage CI run. by @ursg in #975
• Write out vg_f_saved if any VDFs are coming as well. by @ykempf in #976
• Add LFM (Merkin et al 2010) ionosphere solver test. by @ursg in #957
• Pass scancel --full to cancel jobs by @ykempf in #989
• Check script default local by @ykempf in #987
• Patch tracing to spheres at shallow angles. by @ykempf in #970
• Refinement exclusion halfspaces by @ykempf in #985
• Harris update, after #967 by @alhom in #986
• Update ci reference data runner by @markusbattarbee in #991
• Added trap output by @alhom in #977
• Purge dead projects projects/unsupported/* and projects/VelocityBox by @ykempf in #992
• change CI reference data generation job name by @markusbattarbee in #993
• Parallel timer fixes by @lkotipal in #966
• Add a "check cfg files" CI step. by @ursg in #990
• Try to use a cancel job step to scancel by @ykempf in #995
• Add rescaleDensity to Flowthrough to stabilize testing. by @alhom in #994
• Ensure dtIsChanged is false on static timestep by @lkotipal in #997
• Squelch two compiler warnings. by @ursg in #996
• Document refinement region limits behaviour by @ykempf in #998
• Makefile for MPCDF's Viper system by @ursg in #1010
• AMR Testpackage by @lkotipal in #902
• Long overdue fix to the glueChunksTogether.sh script that usually doe… by @ykempf in #1011
• Removed block adjustment in initialization of donotcompute cells. by @ykempf in #1009
• Adding omp simd-pragmas to moment filtering by @veetihaaja in #1002
• No noDipoleInSW by @ykempf in #1005
• Only proc boundary cells in one trans loop by @ykempf in #1008
• Simplify division by kernelSum. by @ykempf in #1018
• Reinstate shrink_to_fit in SpatialCell. by @ykempf in #1016
• Move submodule include paths by @lkotipal in #895
• Vlasiator 5.3.1 dev into master merge by @ykempf in #1025

New Contributors

• @ArnaudLalague made their first contribution in #950
• @astatide made their first contribution in #963
• @veetihaaja made their first contribution in #1002

Full Changelog: v5.3...v5.3.1

This pre-release is a functional first release of Vlasiator with support for NVIDIA GPUs via CUDA and AMD GPUs via HIP/ROCm. The same branch also supports building for CPUs just as before. Slight differences in output show up as a result of differing summation orders in e.g. moments reduction, but testpackage tests all pass with acceptable numerical diffs.

Known issues:

• Block adjustment in acceleration can still be slow.
• VDF-based data reducers are somewhat slow.
• AMD performance is low and requires a mempool for operations (e.g. https://github.com/sfantao/vlasiator-mempool). Planned improvements include support for more robust memory management libraries.
• Field solver and ionospheric grid solver have not yet been ported to GPUs.
• GPU RDMA may not work on all systems, leading to poor MPI performance.
• CUDA compute-sanitizer reports several invalid context errors on launch; these can be ignored.

This introduces the Hashinator library (https://github.com/fmihpc/hashinator) as a required submodule. Make sure to initialize and pull it as well. GPU compilation should use a GPU-compatible compiler for all objects; see e.g. MAKE/Makefile.mahti_cuda for instructions on how to set up a compiler.

New simulation features:

• Actual dynamic AMR based on simulation run-time metrics
• Time-varying inflow conditions for variable solar wind
• Various bugfixes and upgrades to the inner and outer boundary behaviours
• Run-time magnetic fieldline tracing for improved post-processing (such as
  identification of magnetic topology)
• New datareducers such as non-maxwellianity and heat flux

Better user experience:

• New Makefile format with less spammy output
• Significant reduction of compiler warnings
• library build script (build_libraries.sh) included to build all dependencies
• Improved logfile output

Developer notes:

• Development now has a full continuous integration (CI) pipeline
• Header-only library dependencies are handled as git submodules
• Simulation domain decomposition of the fieldsolver grid has had bugfixes and improvements
• Multi-sampled velocity space sampling (nSpaceSamples and nVelocitySamples) has been removed
• Better handling of NaN conditions, using isNaN and isInfinite testing
• New phiprof timer object format thats hould lead to fewer unclosed timer groups

This is a performance update release, containing a more efficient triangular filtering implementation.
This impoves scalability for large 3D runs.

eVlasiator initial 6D pre-release

This release adds support for 3D AMR spatial meshes (introduced in Vlasiator 5.0) and more flexible boundary management.

This includes such highlights as:

• The new ionosphere boundary condition (New Ionosphere implementation #580)
• A custom hashtable for faster velociy space lookups (Replace std::unordered_map with a custom open - bucket fibonacci-hashed hashtable in vmesh. #588)
• Fixed maxwellian inflow conditions (Maxwellian Inflow Fix #582)
• Automatic simulation bailout if the vspace is too small (Vspace wall bailout #587)
• And many more bugfixes!

Note: the old magnetospheric boundary behaviour has been renamed: It is now conductingsphere instead of ionosphere (which now denotes the proper ionosphere solver behaviour and only works for 3D simulations). For spatially 2D setups that retain the old behaviour, config files need to be updated, replacing any occurence of ionosphere by conductingsphere.

Major changes include:

• A long debugging campagin of the AMR solvers, which found many small bugs, so the AMR simulation results are more reliable
• Fixed makefiles for a couple of new machines we have gotten access to
• Scalability and memory usage improvements at scale.
• This code has run some major large-scale runs on supercomputers and a bunch of small ones in the local clusters of the university of helsinki.

eVlasiator pre-release for Global survey

Release used in publication
Electron signatures of reconnection in a global eVlasiator simulation, M. Alho, M. Battarbee, Y. Pfau-Kempf, Yu. V. Khotyaintsev, R. Nakamura, G. Cozzani, U. Ganse, L. Turc, A. Johlander, K. Horaites, V. Tarvus, H. Zhou, M. Grandin, M. Dubart, K. Papadakis, J. Suni, H. George, M. Bussov, and M. Palmroth. Accepted for publication in Geophysical Research Letters. (2022)
https://doi.org/10.1029/2022GL098329

Release used in publication
Vlasov simulation of electrons in the context of hybrid global models: an eVlasiator approach, M. Battarbee, T. Brito, M. Alho, Y. Pfau-Kempf, M. Grandin, U. ganse, K. Papadakis, A. Johlander, L. Turc, M. Dubart, and M. Palmroth. Ann. Geophys. 39, 85–103 (2021)
https://doi.org/10.5194/angeo-39-85-2021