Add script to check if package or its dependencies have changed for tests #9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ofuhrer
reviewed
Oct 27, 2021
Makefile
Outdated
| PULL ?=True | ||
| CONTAINER_ENGINE ?=docker | ||
| RUN_FLAGS ?=--rm | ||
| CHECK_CHANGED_SCRIPT=python changed_from_main.py |
Contributor
There was a problem hiding this comment.
To and old guy like me, this looks a bit scary and I would have a tendency to put this in quotes.
Contributor
There was a problem hiding this comment.
Also, see comment below regarding hash-bang, if that is adopted you could leave the python away here.
Collaborator
Author
There was a problem hiding this comment.
I can add quotes. I would prefer to keep the python here because hash bangs for python make me uncomfortable (do they use your active python env?). Though maybe it should be python3.
Contributor
There was a problem hiding this comment.
That's exactly what #!/usr/bin/env python is for afaik. It picks up the python executable from your modified environment.
See https://discuss.python.org/t/pep-394-clarify-bullet-for-usr-bin-env-python/7953 for example.
Collaborator
Author
There was a problem hiding this comment.
I've updated it to use a shebang, and committed the chmod +x change to the file.
| @@ -0,0 +1,89 @@ | |||
| """ | |||
Contributor
There was a problem hiding this comment.
Coming from a linux/unix environment, I have a tendency to make scripts like these executable and start with a #!/usr/bin/env python3.
Collaborator
Author
There was a problem hiding this comment.
I would prefer to keep the python here because hash bangs for python make me uncomfortable (do they use your active python env?).
| Pace -> driver | ||
| pace -> fv3core | ||
| pace -> physics | ||
| pace -> util |
Contributor
There was a problem hiding this comment.
Do you actually need all of these dependencies explicitly? Or otherwise put, since the dependecy path pace -> fv3core -> util already exists, do we need pace -> util? In any case, we should make sure that for testing we do bottom -> up.
Collaborator
Author
There was a problem hiding this comment.
Yes, pace and driver both directly depend on util, apart from their indirect dependence through other packages.
Contributor
There was a problem hiding this comment.
Yes, I get it. I guess my question is more related to what these dependencies actually imply w.r.t. testing order. Removing these dependencies could favor a bottom-up testing strategy (so we don't test pace right after util, but rather fv3core first).
Collaborator
Author
There was a problem hiding this comment.
All tests which need to run are run simultaneously. These dependencies are not used for testing order and do not imply anything w.r.t. testing order.
| <!-- Pages: 1 --> | ||
| <svg width="230pt" height="260pt" | ||
| viewBox="0.00 0.00 230.24 260.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> | ||
| <svg width="287pt" height="260pt" |
Contributor
There was a problem hiding this comment.
Does it make sense to check-in the *.svg, since it can be out of date w.r.t. to the *.dot file?
Collaborator
Author
There was a problem hiding this comment.
Yes, so that it appears in github's readme. I could add some kind of test that it's not out of date?
Contributor
There was a problem hiding this comment.
Ok, makes sense. I guess it could be auto-generated by some pre-commit hook, but maybe adding a comment in the *.dot file that after modifying it the user should re-generate the *.svg by typing ... is sufficient.
Collaborator
Author
There was a problem hiding this comment.
Added a note along these lines. I'd usually prefer a pre-commit hook, but I would like to avoid breaking pre-commit on systems without dot installed.
Contributor
There was a problem hiding this comment.
I think that makes a lot of sense
Makefile
Outdated
|
|
||
| test_util: | ||
| $(MAKE) -C fv3gfs-util test | ||
| if [ $(shell $(CHECK_CHANGED_SCRIPT) fv3gfs-util) = true ]; then \ |
Contributor
There was a problem hiding this comment.
Using this Makefile logic, how will you make sure that the dependency graph is respected when testing, i.e. making sure that we test bottom -> up on the graph?
Collaborator
Author
There was a problem hiding this comment.
There's no requirement that we test lowest dependencies first. In practice all tests will occur at the same time. Tests failing in util will not necessarily cause a test failure in fv3core.
Contributor
There was a problem hiding this comment.
Hmm... interesting. I would say that if there is a change in util it does not make sense to first test pace (and fail), then test fv3core (and fail) and finally test util (and fail). We should just test util and fail immediately, since there's not point in testing the others in case util already fails.
Collaborator
Author
There was a problem hiding this comment.
But why not test pace, fv3core, and util simultaneously and get failures or not failures at the same time?
Also note to self, I should update this to != false so tests run if the script fails.
Makefile
Outdated
| $(MAKE) -C fv3core savepoint_tests | ||
|
|
||
| savepoint_tests_mpi: | ||
| $(MAKE) -C fv3core savepoint_tests_mpi |
Contributor
There was a problem hiding this comment.
Could do $@ here as well (and above).
mcgibbon
commented
Oct 27, 2021
Makefile
Outdated
|
|
||
| test_util: | ||
| $(MAKE) -C fv3gfs-util test | ||
| if [ $(shell $(CHECK_CHANGED_SCRIPT) fv3gfs-util) = true ]; then \ |
Collaborator
Author
There was a problem hiding this comment.
There's no requirement that we test lowest dependencies first. In practice all tests will occur at the same time. Tests failing in util will not necessarily cause a test failure in fv3core.
changed_from_main.py
Outdated
| DEPENDENCY_PATTERN = re.compile(r"\s*([a-zA-Z0-9]+) -> ([a-zA-Z0-9]+)") | ||
|
|
||
|
|
||
| def get_dependencies() -> Dict[str, Sequence[str]]: |
Collaborator
Author
There was a problem hiding this comment.
Bug: this currently only retrieves direct dependencies and not nested dependencies. I'll fix it.
Makefile
Outdated
| PULL ?=True | ||
| CONTAINER_ENGINE ?=docker | ||
| RUN_FLAGS ?=--rm | ||
| CHECK_CHANGED_SCRIPT=python changed_from_main.py |
Collaborator
Author
There was a problem hiding this comment.
I can add quotes. I would prefer to keep the python here because hash bangs for python make me uncomfortable (do they use your active python env?). Though maybe it should be python3.
| @@ -0,0 +1,89 @@ | |||
| """ | |||
Collaborator
Author
There was a problem hiding this comment.
I would prefer to keep the python here because hash bangs for python make me uncomfortable (do they use your active python env?).
| Pace -> driver | ||
| pace -> fv3core | ||
| pace -> physics | ||
| pace -> util |
Collaborator
Author
There was a problem hiding this comment.
Yes, pace and driver both directly depend on util, apart from their indirect dependence through other packages.
| <!-- Pages: 1 --> | ||
| <svg width="230pt" height="260pt" | ||
| viewBox="0.00 0.00 230.24 260.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> | ||
| <svg width="287pt" height="260pt" |
Collaborator
Author
There was a problem hiding this comment.
Yes, so that it appears in github's readme. I could add some kind of test that it's not out of date?
twicki
reviewed
Nov 4, 2021
| @@ -0,0 +1,89 @@ | |||
| """ | |||
| This script determines whether one of the projects in the repo or any of its | |||
Contributor
There was a problem hiding this comment.
should this really be in the top level directory?
In fv3core we did use a .jenkins folder for these type of things as it is pretty CI related. Or we could put this into a util folder?
Collaborator
Author
There was a problem hiding this comment.
If our top level directory gets cluttered I could see moving this eventually, but this is an integral part of running the top-level Makefile targets as we're currently planning to use it. If we call it directly in Jenkins instead of calling it inside the Makefile (as I did for test_util), we can move it then.
changed_from_main.py
Outdated
| import os | ||
| import subprocess | ||
| from typing import Dict, Sequence | ||
| import sys |
Collaborator
Author
There was a problem hiding this comment.
Fixed. I've gotten to a point of assuming all my imports are used because pre-commit will catch it, and forgot pre-commit isn't set up yet.
twicki
reviewed
Nov 4, 2021
| util [shape=oval] | ||
| pace [shape=box] | ||
| fv3core [shape=oval, label="fv3core"] | ||
| driver [shape=oval, label="driver"] |
Contributor
There was a problem hiding this comment.
driver does not exist yet. should it already be here?
Collaborator
Author
There was a problem hiding this comment.
No harm in including it because it's not depended on by any other components, it can happen now or when the folder gets added. I believe it will be added as part of Rhea's current PR(s).
Contributor
There was a problem hiding this comment.
it's fine if it is here. Right now it exits not very gracefully if you call changed_from_main.py driver but this is very temporary. If it is not blocking any CI related work this is totally fine!
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwidth Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustment threshold to const * Remove unused if leading to empty code block * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Add openmpi to the image * Fix unit tests (remove dxa, dya rely on halo ex) * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Deactivate relocatability test due to Python crash Logged as issue 16 * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> --------- Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Devops/GitHub actions on (ai2cm#15) * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Fix non-deterministic temporaries by using `zeros` everywhere instead of `empty` * Update dsl/pace/dsl/caches/codepath.py Co-authored-by: Oliver Elbert <[email protected]> * Refactor the test to go around so reload bug --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> --------- Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Lint --------- Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) * Update fv3core/pace/fv3core/initialization/geos_wrapper.py Co-authored-by: Oliver Elbert <[email protected]> * FVTP2D: somewhat better workaround --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> --------- Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Lint --------- Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- --------- * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- * Lint --------- * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) * NASA fork sync. (ai2cm#37) (ai2cm#30) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- --------- * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- * Lint --------- * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) --------- --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
twicki
pushed a commit
to twicki/pace
that referenced
this pull request
Aug 22, 2025
* NASA commits sync (ai2cm#31) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> --------- Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Lint --------- Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) * NASA fork sync. (ai2cm#37) (ai2cm#30) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * lint * More linting * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings Lint * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" This reverts commit 4fc5b4d. * Revert "Remove previous per stencil override of default_build_folder" This reverts commit 2245027. * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Replace all logger with pace_log Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Distributed compilation on orchestrated backend for NxN layouts (ai2cm#14) * Adapt orchestration distribute compile for NxN layout * Remove debug code * Add a more descriptive string base postfix for cache naming Identify the code path for all cases Consistent reload post-compile Create a central space for all caches generation logic No more original layout check required * Add a test on caches relocatability * Verbose todo * Linting on PR * Run main unit test * Update python to available 3.8.12 * Remove cd to pace * Lint: git submodule recursive * Typo * Add openmpi to the image * Linting * Fix unit tests (remove dxa, dya rely on halo ex) * typo * Change name of jobs * Missing enum * Lint imports * Fix unit tests * Deactivate relocability test due to Python crash Logged as issyue 16 * Typo * Raise for 1,X and X,1 layouts which requires a new descriptor * Added ak, bk for 137 levels in eta.py * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Log info GEOS bridge (ai2cm#18) * Add floating point precision to GEOS bridge init * lint * Add device PCI bus id (for MPS debug) * Typo + lint * Try to detect MPS reading the "log" pipe * Lint * Clean up * Update geos/develop to grab NOAA PR9 results (ai2cm#21) * Verbose choice of block/grid size * added build script for c5 * updated repo to NOAA * GEOS integration (ai2cm#9) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Remove unused if leading to empty code block * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 Fix bad merge for bdt with GEOS_Wrapper * Remove unused code * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Revert "Remove previous per stencil override of default_build_folder" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Fix bad requirements syntax * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Fix or explain inlined import * Verbose runtime error when bad dt_atmos * Verbose warm up * re-initialize heat_source and diss_est each call, add do_skeb check to accumulation --------- --------- * [NOAA:Update] Bring back ai2cm#15 & doubly periodic domain (ai2cm#25) * Feature/dp driver (ai2cm#13) * initial commit * adding test config * adding the rest of driver and util code * updating history.md * move u_max to dycore config * uncomment assert * added comment explaining the copy of grid type to dycore config * Turn main unit test & lint on PR, logger clean up [NASA:Update] (ai2cm#15) * Initialize GeosDycoreWrapper with bdt (timestep) * Use GEOS version of constants * 1. Add qcld to the list of tracers beings advected 2. Made GEOS specific changes to thresholds in saturation adjustment * Accumulate diss_est * Allow GEOS_WRAPPER to process device data * Add clear to collector for 3rd party use. GEOS pass down timings to caller * Make kernel analysis run a copy stencil to compute local bandwith Parametrize tool with backend, output format * Move constant on a env var Add saturation adjustement threshold to const * Restrict dace to 0.14.1 due to a parsing bug * Add guard for bdt==0 * Fix theroritical timings * Fixed a bug where pkz was being calculated twice, and the second calc was wrong * Downgrade DaCe to 0.14.0 pending array aliasing fix * Set default cache path for orchestrated DaCe to respect GT_CACHE_* env * Remove previous per stencil override of default_build_folder * Revert "Set default cache path for orchestrated DaCe to respect GT_CACHE_* env" * Read cache_root in default dace backend * Document faulty behavior with GT_CACHE_DIR_NAME * Check for the string value of CONST_VERSION directly instead of enum * Protect constant selection more rigorusly. Clean abort on unknown constant given * Log constants selection * Refactor NQ to constants.py * Introduce PACE_LOGLEVEL to control log level from outside * Code guidelines clean up * Devops/GitHub actions on (ai2cm#15) * Linting on PR * Run main unit test * Update python to available 3.8.12 * Fix unit tests (remove dxa, dya rely on halo ex) * Update HISTORY.md * Adapt log_level in driver.run * Verbose the PACE_CONSTANTS * Doc log level hierarchical nature --------- * Lint --------- * Update gt4py, dace, cleanup (ai2cm#19) * Update gt4py to top of master on June 21 * Update DaCe to 0.14.2 Workaround aliasing issue in FiniteVolumeTransport * Fix to gt4py storage * Downgrade to dace 0.14.1 * DaCe to 0.14.4 Orchestrating NonHydrostaticPressureGradient Adptating code to newer gt4py * Regenerate constraints.txt * Default constants to GFS Fix snapshot for GPU runs Lint on ETA Fix log level * Remove `daint_venv` submodule * Adding dace as a submodule Removing buildenv as a submodule * Update gt4py to latest master * Skip ConstantPropagation during `Simplify` * Remove buidlenv * Update requirements_dev.txt * Add editable util to requirements_dev.txt * lint * scipy for tests is now needed * Pin `DaCe` to pace-fixes-0 merge * Remove logging setup in test_translate * Make cupy import robust to device not being available * Fix to GEOS bridge MPS detection * Up gt4py to August 14th EOD: - Hip/ROCm - New allocators * DaCE module: swap SSH for HTTPS (ai2cm#26) * GEOS GridTools stencils build override (ai2cm#27) * Stencil build override for GEOS * Deactivate warnings if PACE_LOGLEVEL is > WARNING * Better log level * Bad merge (again) --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> * Update DaCe to 0.15.1 RC (ai2cm#35) - Update: DaCe to 0.15.1 RC and GT4Py to latest main - Minor: orchestration build logging - Minor: dead code clean up --------- Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Purnendu Chakraborty <[email protected]> Co-authored-by: Rusty Benson <[email protected]> Co-authored-by: Oliver Elbert <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
mainbranchmake test_utiltarget to only run if it has changed compared to the main branch