-
Notifications
You must be signed in to change notification settings - Fork 121
Information Geometric Regularization #900
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The complete WENO test suite now passes on CPUs with the IGR code added to the codebase. 2D Sedov blast waves with 1 and 2 fluids run on 1 and 4 ranks with IGR on CPUs.
All tests still pass on CPUs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't do a close read so I can't speak to the math being right (I'll leave that up to others and the tests), but I did flag a couple portability issues that would be good to address before this gets merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i added a comment on the helper file. also can you add to the readme IGR capability.
|
can merge after that |
User description
Description
This PR adds the basic functionality of IGR to the code base. It includes 3rd and 5th order reconstruction, viscous and inviscid simulations, 2D and 3D simulations, Jacobi and Gauss-Seidel iterative solves, and an optional pressure limiter. It does not include more advanced features like unified memory or downsampling of I/O files.
Note (6/21/25, 5pm) - The current implementation matches the GB code performance on A100 GPUs. The performance of this implementation is still lacking on AMD hardware.
Note (7/6/25) - The update Frontier implementation is faster, and new traces have been added. The grindtime of the implementation in this code for a 300^3 case is 0.84. The grind time for the GB code with the same problem size was 0.64.
Note (7/8/25) - Frontier performance on this branch is now better than the BG code, it seems? 1.70 seconds on the GB code versus 1.33 seconds on this code? New hip traces have been added for both the GB code and this code for a 700^3 case.
Type of change
Please delete options that are not relevant.
Scope
How Has This Been Tested?
2D Triple Point - This simulation was run using the
examples/2D_IGR_TriplePointcase file and ran on A100 GPUs.https://github.com/user-attachments/assets/dabba064-7c44-46e0-a85a-246e790a79bf
2D 2 Fluid Blast Wave - This simulation was run using the
examples/2D_IGR_2fluidcase file and ran on A100 GPUs.https://github.com/user-attachments/assets/f0a8036a-2560-49fd-9778-aa02a7c72679
3D Taylor-Greene Vortex - This simulation was run using the
examples/3D_IGR_TaylorGreenVortexcase file and ran on A100 GPUs.https://github.com/user-attachments/assets/3af5548e-c11b-46b0-a642-1a423ac55de2
3D Jet - This simulation was run using the
examples/3D_IGR_jetcase file and ran on A100 GPUs.https://github.com/user-attachments/assets/be985655-c369-43d4-9631-cd8dd98a1719
Checklist
docs/)examples/that demonstrate my new feature performing as expected.They run to completion and demonstrate "interesting physics"
./mfc.sh formatbefore committing my codeIf your code changes any code source files (anything in
src/simulation)To make sure the code is performing as expected on GPU devices, I have:
nvtxranges so that they can be identified in profiles./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PRCurrent implementation summary:
newNV.csv
GB Implementation summary (warm start iteration count is different)
oldNV.csv
./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.Current Implementation Trace:
newnewFrontier.csv
GB Implementation Trace:
oldoldFrontier.csv
PR Type
Enhancement
Description
• Implements Information Geometric Regularization (IGR) as a new numerical method for fluid dynamics simulations
• Adds comprehensive IGR solver with 3rd and 5th order reconstruction, supporting both viscous and inviscid flows
• Integrates IGR throughout the entire codebase including simulation, pre-processing, and post-processing modules
• Provides IGR-specific boundary conditions, time stepping, and iterative solvers (Jacobi and Gauss-Seidel)
• Adds extensive test cases and example configurations for 2D and 3D IGR simulations
• Includes performance optimizations and GPU support for IGR computations
• Updates documentation with IGR parameter descriptions and methodology references
• Maintains backward compatibility with existing WENO-based solver functionality
Changes walkthrough 📝
31 files
m_rhs.fpp
Integration of Information Geometric Regularization solversrc/simulation/m_rhs.fpp
• Added
use m_igrmodule import for Information GeometricRegularization functionality
• Wrapped existing finite volume solver
code with conditional
.not. igrchecks• Added IGR-specific code paths
including
s_igr_riemann_solver,s_igr_iterative_solve, ands_igr_sigma_xcalls• Modified memory allocation patterns to support
both IGR and traditional finite volume approaches
m_time_steppers.fpp
Time stepper modifications for IGR supportsrc/simulation/m_time_steppers.fpp
• Added conditional allocation of primitive variables based on
igrflag
• Modified RHS variable allocation to use extended bounds for IGR
(
-1:m+1,-1:n+1,-1:p+1)• Updated run-time information calls to use
conservative variables for IGR instead of primitive variables
• Added
IGR-specific enthalpy computation calls in time stepping routines
m_boundary_common.fpp
IGR boundary condition support implementationsrc/common/m_boundary_common.fpp
• Added new
jac_sfscalar field declaration for Jacobian handling•
Implemented
s_populate_F_igr_bufferssubroutine for IGR boundarycondition handling
• Added support for periodic, reflective, and
default boundary conditions for IGR Jacobian field
m_initial_condition.fpp
Conservative variable allocation bounds updatesrc/pre_process/m_initial_condition.fpp
• Modified conservative variable allocation to use extended buffer
bounds instead of domain bounds
• Changed from
(0:m, 0:n, 0:p)to(idwbuff(1)%beg:idwbuff(1)%end, idwbuff(2)%beg:idwbuff(2)%end,idwbuff(3)%beg:idwbuff(3)%end)m_start_up.fpp
Integrate IGR module initialization and finalizationsrc/simulation/m_start_up.fpp
• Added IGR module import and initialization/finalization logic
•
Modified module initialization order to conditionally use IGR or WENO
modules
• Updated grind time calculation with proper type casting
•
Added IGR-specific GPU variable updates
m_mpi_common.fpp
Update MPI topology for IGR reconstruction order supportsrc/common/m_mpi_common.fpp
• Added reconstruction order variable to handle both WENO and IGR
orders
• Updated processor topology calculations to use generic
reconstruction order
• Modified error messages to reference both WENO
and IGR orders
m_checker.fpp
Add IGR input validation and compatibility checkssrc/simulation/m_checker.fpp
• Added comprehensive IGR input validation function
• Reorganized
input checking to conditionally validate IGR or WENO parameters
•
Added IGR compatibility checks with other features
m_mpi_proxy.fpp
Add MPI support for IGR parameterssrc/simulation/m_mpi_proxy.fpp
• Added MPI broadcasting for IGR-related parameters
• Included new IGR
variables in MPI communication arrays
m_checker_common.fpp
Add common IGR validation and improve cell countingsrc/common/m_checker_common.fpp
• Added IGR input validation function with order and grid size checks
• Reorganized common input checking to conditionally validate IGR
•
Updated total cells calculation with proper integer types
m_sim_helpers.fpp
Implement IGR-specific variable conversion logicsrc/simulation/m_sim_helpers.fpp
• Added IGR-specific variable extraction logic for density and volume
fractions
• Modified velocity and energy calculations for IGR
compatibility
• Updated primitive variable handling based on IGR vs
WENO modes
m_variables_conversion.fpp
Update variable conversion for IGR supportsrc/common/m_variables_conversion.fpp
• Added IGR-specific alpha_rho and alpha extraction logic
• Modified
variable conversion to handle IGR's different variable organization
•
Updated conservative variable handling for IGR compatibility
3dHardcodedIC.fpp
Add 3D IGR jet hardcoded initial conditionsrc/pre_process/include/3dHardcodedIC.fpp
• Added new hardcoded initial condition case 302 for 3D IGR jet
simulation
• Implemented jet flow field initialization with smooth
transitions
m_start_up.f90
Add IGR support to post-processing startupsrc/post_process/m_start_up.f90
• Added IGR parameter reading in input file processing
• Modified
alpha field calculation for IGR compatibility in post-processing
m_global_parameters.fpp
Add IGR global parameters and system configurationsrc/pre_process/m_global_parameters.fpp
• Added IGR global parameters and initialization
• Modified system
size calculation for IGR's different variable organization
• Updated
coordinate bounds configuration to include IGR parameter
m_global_parameters.fpp
Add IGR parameters to post-processing globalssrc/post_process/m_global_parameters.fpp
• Added IGR parameters to post-processing global parameters
• Modified
system size calculation for IGR compatibility
m_data_output.fpp
Disable primitive variable output for IGR modesrc/simulation/m_data_output.fpp
• Disabled primitive variable writing for IGR mode in 1D, 2D, and 3D
cases
• Added conditional checks to prevent incompatible output
formats with IGR
m_data_output.fpp
Update data output for IGR boundary conditionssrc/pre_process/m_data_output.fpp
• Modified boundary condition file writing to use conservative
variables for IGR
• Updated data file writing to use proper array
bounds
m_helper_basic.fpp
Update coordinate bounds for IGR buffer requirementssrc/common/m_helper_basic.fpp
• Updated coordinate bounds configuration to handle IGR-specific
buffer sizes
• Added IGR parameter to buffer size calculation function
m_start_up.fpp
Add IGR parameter support to pre-processing startupsrc/pre_process/m_start_up.fpp
• Added IGR parameter reading in input file processing
• Updated
global cell count calculation with proper integer types
m_mpi_proxy.fpp
Add IGR MPI support to post-processingsrc/post_process/m_mpi_proxy.fpp
• Added IGR parameter to MPI broadcasting in post-processing
m_constants.fpp
Add IGR default constantssrc/common/m_constants.fpp
• Added default constants for IGR iteration parameters
• Defined
default values for IGR solver configuration
m_patch_helper.fpp
Add patch helper module with smooth transition functionssrc/pre_process/m_patch_helper.fpp
• Added new module with smooth cutoff functions for patch
initialization
• Implemented mathematical functions for smooth
transitions in initial conditions
m_mpi_proxy.fpp
Add IGR MPI support to pre-processingsrc/pre_process/m_mpi_proxy.fpp
• Added IGR parameter to MPI broadcasting in pre-processing
m_patches.fpp
Import patch helper modulesrc/pre_process/m_patches.fpp
• Added patch helper module import for smooth transition functions
case.py
Add 3D IGR jet example caseexamples/3D_IGR_jet/case.py
• Added complete 3D jet simulation case configuration for IGR
•
Configured domain, fluid properties, and IGR-specific parameters
case.py
Add 2D IGR triple point example caseexamples/2D_IGR_triple_point/case.py
• Added 2D triple point shock interaction case for IGR validation
•
Configured multi-fluid setup with IGR parameters
case.py
Add IGR benchmark casebenchmarks/igr/case.py
• Added IGR benchmark case for performance testing
• Configured
Taylor-Green vortex problem with viscous IGR
case.py
Add 2D IGR two-fluid example caseexamples/2D_IGR_2fluid/case.py
• Added 2D two-fluid blast wave case for IGR testing
• Configured
multi-fluid setup with pressure jump
case.py
Add 3D IGR Taylor-Green vortex example caseexamples/3D_IGR_TaylorGreenVortex/case.py
• Added 3D Taylor-Green vortex case for IGR validation
• Configured
viscous flow with analytical initial conditions
case.py
Add IGR support to case configuration systemtoolchain/mfc/case.py
• Added IGR parameter handling in case configuration
• Updated WENO
order logic to accommodate IGR settings
• Added IGR-specific
preprocessor definitions
case_dicts.py
Add IGR parameters to case dictionary systemtoolchain/mfc/run/case_dicts.py
• Added IGR parameters to case dictionary definitions
• Updated case
optimization parameters to include IGR settings
2 files
m_global_parameters.fpp
Global parameter additions for IGR configurationsrc/simulation/m_global_parameters.fpp
• Added IGR-related parameters:
igr_iter_solver,igr_order,igr,igr_pres_lim, andviscous• Modified system size calculation to
account for IGR with different fluid configurations
• Added
initialization of IGR parameters with default values
• Updated
coordinate bounds configuration to include IGR parameter
frontier.mako
Update Frontier template for IGR requirementstoolchain/templates/frontier.mako
• Added unlimited stack size setting for IGR computations
• Added
unbuffered output flag for interactive runs
2 files
m_surface_tension.fpp
Fix GPU loop macro in surface tension modulesrc/simulation/m_surface_tension.fpp
• Fixed GPU loop macro usage for proper parallelization
m_checker.fpp
Add precision validation for output formatsrc/post_process/m_checker.fpp
• Added precision validation to prevent double precision output with
single precision build
26 files
cases.py
Add IGR test case generation and GPU filteringtoolchain/mfc/test/cases.py
• Added IGR test case generation with different orders and solvers
•
Integrated IGR testing into existing test framework
• Added
GPU-specific test filtering for Gauss-Seidel solver
test.py
Add GPU test filtering for IGR solverstoolchain/mfc/test/test.py
• Added GPU-specific test filtering for Gauss-Seidel solver
golden-metadata.txt
Add IGR test metadata filetests/9F4E47F1/golden-metadata.txt
• Added test metadata file for IGR test case
golden-metadata.txt
Add IGR test metadata filetests/0045D9F8/golden-metadata.txt
• Added test metadata file for IGR test case
golden-metadata.txt
Add IGR test metadata filetests/238A7B1F/golden-metadata.txt
• Added test metadata file for IGR test case
golden-metadata.txt
Add IGR test metadata filetests/BF8FDC2A/golden-metadata.txt
• Added test metadata file for IGR test case
golden-metadata.txt
Add IGR test metadata filetests/023DC5B9/golden-metadata.txt
• Added test metadata file for IGR test case
golden-metadata.txt
Add test metadata for IGR test case 106C0BE6tests/106C0BE6/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components (pre_process,
simulation, post_process, syscheck, documentation)
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 18A71AB2tests/18A71AB2/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 2699E9DFtests/2699E9DF/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 4440D46Btests/4440D46B/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 9EE432D4tests/9EE432D4/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case A6DB36C2tests/A6DB36C2/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case AE3FC5CBtests/AE3FC5CB/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case C7A6B609tests/C7A6B609/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case E49EF7B6tests/E49EF7B6/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case E8979E4Atests/E8979E4A/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case EB58AF7Ftests/EB58AF7F/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 37DDE283tests/37DDE283/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case 6077374Ftests/6077374F/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case A57E30FEtests/A57E30FE/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case AFBCBDFAtests/AFBCBDFA/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case CEAF553Atests/CEAF553A/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case D49DCFC0tests/D49DCFC0/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case D7885E17tests/D7885E17/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
golden-metadata.txt
Add test metadata for IGR test case F44A293Atests/F44A293A/golden-metadata.txt
• Added new test metadata file with build configuration details
•
Contains CMake configuration for all MFC components
• Records system
information including Apple M2 CPU details and compiler versions
2 files
references.md
Add IGR methodology referencedocs/documentation/references.md
• Added reference to IGR methodology paper by Cao and Schäfer
case.md
Document IGR configuration parameters and fix typodocs/documentation/case.md
• Added documentation for IGR (Information Geometric Regularization)
parameters
• Added 6 new IGR-related configuration options including
igr,igr_order,alf_factor, etc.• Fixed typo in
radiusparameter (wasradiue)36 files