[Experimental] Debugger #137

FlorianDeconinck · 2025-04-28T16:20:02Z

Description
Embryo of a gt backend purposed debugger.
Feature list:

Can save netcdf for each parameter IN and OUT by name (no read/write analysis, just name matching on parameters list of stencils and instrumented methods)
Save all IN and OUT of stencils and instrumented methods

System is multi-rank and multi-call ready. It is driven by a yaml file of the shape:

mode: Runtime
stencils_or_class:
  - copy_corners_x_nord
  - copy_corners_y_nord
  - fy_calc_stencil_column
track_parameter_by_name:
  - fy

How Has This Been Tested?
This has been used during f32 debugging of the Dynamics

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
Any dependent changes have been merged and published in downstream modules
New check tests, if applicable, are included

Verbose code

Check only when debugger present

…a field

…/debugger

Better handling of types unknown to xarray

romanc

Looks good to me. Couple small notes and questions inline, but nothing that would make me formally request changes.

I mostly skimmed over the viz/fv3 part since that was just copied from back then.

Thanks for adding this. I think this will be add a lot to quality of live and we'll probably come back later (once we used it a bit) to make changes.

ndsl/viz/fv3/README.md

ndsl/viz/fv3/_plot_cube.py

ndsl/debug/config.py

ndsl/debug/debugger.py

ndsl/debug/config.py

ndsl/debug/mode.py

ndsl/debug/tooling.py

Add warning for config not existing but set Use Pathlib for cross-OS resilience

romanc

Found one debug print statement, which I must have missed in my earlier review. Otherwise all good now.

ndsl/viz/fv3/_plot_cube.py

oelbert

I have a small question on the implementation of debugger, and a (potentially) larger one on the license for the viz code

ndsl/debug/debugger.py

ndsl/dsl/stencil.py

oelbert · 2025-05-20T15:37:38Z

ndsl/viz/fv3/README.md

@@ -0,0 +1,14 @@
+# Acknowledgment
+
+This code was lifted from <https://github.com/ai2cm/fv3net> and developped by AI2 under the MIT license (see below).


We're sure there's not a license violation or conflict between this and our top-level license?

Also assuming you talked to the AI2 folks about this?

Actually, why not directly import fv3net? I have it as a Pace dependency (at least in the dockerfile...)?

Because fv3net is a huge repository - and the viz is a very small code untouched for the last three years. It's the case of bring in thousands of line of code, for the benefit of using a handful.

I'll check the license, I think it's ok.

Not a layer here, but the MIT license (as stated below) is pretty loose. In particular it allows to modify and re-distribute the code provided that the license header is preserved.

So I checked and this is correct. If we were to make "substantial" changes we could rope the code in the Apache 2.0.

MIT is basically covering up AI2 for any side effect and by knock-on free us to use or reuse as is. If we modify we can argue that the license applying to the code is the one under we operate

I also checked lol, but yeah it seems like this is fine. Probably still good to drop Oli WM a line to let him know if you haven't already

ndsl/debug/config.py

oelbert

Awesome, thanks!

FlorianDeconinck added 10 commits April 15, 2025 16:08

Save in/out of stencils and decorated functions

b8c44cb

Move instrument to debugger

0a379d8

Verbose code

Log debugger on & config in debug

95ab6aa

Move @Instrument once more

530208c

Add track_by_parameter and moved results in subdirectory

05668c7

Simplify configuration

ac000d8

Minor fix to filename

f8dfbd9

Fix stencils save

2c43481

Check only when debugger present

Normalize dims naming

c5d02dc

Use full qualified name when possible

454a898

FlorianDeconinck requested a review from oelbert April 28, 2025 16:20

FlorianDeconinck and others added 16 commits April 28, 2025 12:20

Merge branch 'develop' into experimental/debugger

1de97cf

Lint

440b5ca

Coarse documentation for debugger config

c2684e2

Protect catch-all for in xarray failures that don't raise properly

fa81043

Config: allow for compute domain save in Quantity instead of full dat…

f1afbb1

…a field

Added tools to plot pyFV3 data

2e8680e

removed print statements

92d0557

renamed folders

a16ade6

Merge remote-tracking branch 'personal/debug_tools' into experimental…

ac9e105

…/debugger

plot_cube_sphere public API

d5f77af

FV3 README license ackgnoledgement

ea518dd

Debugger knows how to serialize dataclasses

f8fe74b

Better handling of types unknown to xarray

Restore Robinson projection

ee4775e

Lint

a23ce03

Moar lint

a7351c7

Merge branch 'develop' into experimental/debugger

368803c

FlorianDeconinck requested review from fmalatino and romanc May 19, 2025 13:15

fmalatino previously approved these changes May 19, 2025

View reviewed changes

romanc previously approved these changes May 19, 2025

View reviewed changes

FlorianDeconinck added 3 commits May 19, 2025 14:56

Remove DebugMode

379bdfd

Add warning for config not existing but set Use Pathlib for cross-OS resilience

Lint

ceb6d6c

cartopy import is not optional

dda1013

FlorianDeconinck dismissed stale reviews from romanc and fmalatino via dda1013 May 19, 2025 18:57

FlorianDeconinck added 2 commits May 19, 2025 15:04

Clear __init__

8ca571d

lint

5a2f376

FlorianDeconinck requested review from fmalatino and romanc May 19, 2025 19:25

fmalatino previously approved these changes May 20, 2025

View reviewed changes

romanc previously approved these changes May 20, 2025

View reviewed changes

ndsl/viz/fv3/_plot_cube.py Outdated Show resolved Hide resolved

Remove debug print

45ace08

FlorianDeconinck dismissed stale reviews from romanc and fmalatino via 45ace08 May 20, 2025 12:39

FlorianDeconinck requested a review from fmalatino May 20, 2025 14:35

Merge branch 'develop' into experimental/debugger

ca9e105

oelbert reviewed May 20, 2025

View reviewed changes

FlorianDeconinck added 2 commits May 20, 2025 11:42

Use field instead of view[:]

d28198e

Spelling

2595438

FlorianDeconinck requested review from oelbert and romanc May 20, 2025 17:09

romanc approved these changes May 20, 2025

View reviewed changes

fmalatino approved these changes May 20, 2025

View reviewed changes

oelbert approved these changes May 20, 2025

View reviewed changes

FlorianDeconinck merged commit 2a6150c into NOAA-GFDL:develop May 20, 2025
5 checks passed

FlorianDeconinck deleted the experimental/debugger branch May 20, 2025 17:49

		@@ -0,0 +1,14 @@
		# Acknowledgment

		This code was lifted from <https://github.com/ai2cm/fv3net> and developped by AI2 under the MIT license (see below).

[Experimental] Debugger #137

[Experimental] Debugger #137

Uh oh!

Conversation

FlorianDeconinck commented Apr 28, 2025

Uh oh!

romanc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

romanc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

oelbert May 20, 2025

Choose a reason for hiding this comment

Uh oh!

oelbert May 20, 2025

Choose a reason for hiding this comment

Uh oh!

FlorianDeconinck May 20, 2025

Choose a reason for hiding this comment

Uh oh!

romanc May 20, 2025

Choose a reason for hiding this comment

Uh oh!

FlorianDeconinck May 20, 2025

Choose a reason for hiding this comment

Uh oh!

oelbert May 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants