Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Agentic] Move array hashing into a shared _combine_common substrate - #7276

Open
bjlittle wants to merge 4 commits into
SciTools:greenfieldfrom
bjlittle:pr1-hashing-substrate
Open

[Agentic] Move array hashing into a shared _combine_common substrate#7276
bjlittle wants to merge 4 commits into
SciTools:greenfieldfrom
bjlittle:pr1-hashing-substrate

Conversation

@bjlittle

Copy link
Copy Markdown
Member

πŸ€– Agentic pull request

This change was written by Claude (Opus 5), driven by @bjlittle. Commits carry a
Co-Authored-By trailer. Please review it as you would any other contribution β€”
and more sceptically, if that is your inclination.

What this does

Moves the array-hashing machinery out of lib/iris/_concatenate.py and into a
new private module, lib/iris/_combine_common.py.

This is row 1 of the roadmap in the merge and concatenate design spec (#7274),
and implements Β§5.1 of that spec. Nothing else in the programme lands until this
one is reviewed β€” it is deliberately the smallest possible first step, and it is
as much a test of whether the approach is worth continuing as it is a change.

The step-by-step plan this follows landed separately in #7275, and is at
docs/src/developers_guide/plans/2026-09-10-hashing-substrate.md if you want the
provenance. You should not need it to review this: the plan is a record of how
the change was arrived at, not an argument for it.

The hashing layer was added for concatenate in #5926. Merge needs exactly the
same capability, and today cannot have it without importing from
_concatenate.py β€” which _concatenate.py cannot offer, because it imports
iris.cube. The new module imports nothing from Iris at runtime, so either
engine can use it with no risk of a circular import. test_imports.py enforces
that by walking the module's AST, and a second test guards the guard so the
first cannot pass by failing to look.

This is a relocation, not a rewrite

No behaviour changes. The moved code is identical to the original apart from
dropping the leading underscore on seven names, plus three lines noted below.
Two checks you can run yourself:

1. The moved block is unchanged apart from the renames:

git show upstream/greenfield:lib/iris/_concatenate.py | sed -n '305,541p' \
  | sed -e 's/\b_hash_ndarray\b/hash_ndarray/g' -e 's/\b_hash_chunk\b/hash_chunk/g' \
        -e 's/\b_hash_aggregate\b/hash_aggregate/g' -e 's/\b_hash_array\b/hash_array/g' \
        -e 's/\b_ArrayHash\b/ArrayHash/g' -e 's/\b_array_id\b/array_id/g' \
        -e 's/\b_compute_hashes\b/compute_hashes/g' > /tmp/expected.py
sed -n '/^def hash_ndarray/,$p' lib/iris/_combine_common.py > /tmp/actual.py
diff -u /tmp/expected.py /tmp/actual.py

Output is two hunks, three lines β€” the de-shadowing described below.

2. No concatenate test was touched. test_hashing.py moves to the new
package unchanged but for its import lines; no other test under
lib/iris/tests/unit/concatenate/ is edited.

git diff upstream/greenfield --stat -- lib/iris/tests/unit/concatenate/

Where to actually look

Everything above is mechanical. The only part that needed judgement is the
array_id rename, in two places:

  • Five call sites in _concatenate.py assigned to a local named
    array_id. Once the imported function has that name, the local shadows it and
    the next line raises UnboundLocalError. The locals are now points_id and
    bounds_id, which is what they always meant.
  • Three bindings inside compute_hashes itself had the same collision. It is
    inert β€” no scope in that module calls the function β€” but it is the same trap,
    sitting in the module that defines the name. They are now _ and key.

Neither ruff nor mypy catches this class of shadowing, so it is worth a human
glance.

An open question for the reviewer

Dropping the underscore prefix makes this a move and a rename, and that is a
decision I would rather you took than I did.

The case for it: these names are now a small internal API consumed by two
modules, and the underscore no longer says anything the module's own _ prefix
does not already say.

The case against: it turns a diff you could verify by eye into one you have to
read, and it is the sole cause of all eight local renames above.

If you would rather review a pure git mv with the underscores kept, say so
and I will drop the rename.
The module's location is what the rest of the
programme depends on; the spelling of the names is not, and I have no stake in
it.

Testing

pytest -n auto lib/iris/tests/unit gives 7072 passed against 7070 on
greenfield β€” the two additions are the new import-invariant tests. The one
pre-existing failure (test__chunk_control.py::test_netcdf_v3) and ten
pre-existing errors are unchanged by this branch and unrelated to it.

bjlittle and others added 3 commits September 11, 2026 10:07
Lifts the array-hashing machinery out of _concatenate.py into a new private
module, lib/iris/_combine_common.py, so that _merge.py can reach it without
importing _concatenate.py.  The substrate imports nothing from Iris at
runtime; the coordinate annotations on array_id sit behind TYPE_CHECKING.

Names lose their underscore prefix on the move, the module itself already
being private.  The two local variables that would otherwise have shadowed
the imported array_id are renamed to points_id and bounds_id.

Pure relocation: no behaviour change.  Row 1 of the roadmap in
docs/src/developers_guide/specs/2026-09-10-merge-concatenate-design.md.

Co-Authored-By: Claude Opus 5 <[email protected]>
`iris._combine_common` sits beneath both `iris._merge` and
`iris._concatenate`, so either engine must be able to import it with no
risk of a circular import.  That property holds today by inspection, but
nothing enforces it, and the merge side has yet to be written -- by the
time the invariant is load-bearing, the commit that broke it would be
long merged.

Parse the module and walk its AST, collecting every module imported
outside an `if TYPE_CHECKING:` block, and assert that none of them are
Iris.  A second test guards the guard, checking that the visitor really
does distinguish a runtime import from an annotation-only one, so the
first test cannot pass by failing to look.

Co-Authored-By: Claude Opus 5 <[email protected]>
Dropping the underscore from `_array_id` put the function on the same
name as three local bindings inside `compute_hashes`, which previously
sat alongside it without collision.  Nothing in either scope calls the
function, so the shadowing is harmless today, but it is precisely the
trap that had to be defused on the `_concatenate.py` side of the move,
and leaving it in the module that defines the name invites the next
reader to fall into it.

Rename the unused element of the `group_key` unpack to `_`, and the loop
variable in `compute_hashes` to `key`.  Behaviour is unchanged: these are
the only three lines in which the relocated code differs from the
original modulo the seven renames.

Co-Authored-By: Claude Opus 5 <[email protected]>
Add the towncrier fragment, move roadmap row 1 to "in progress", and
cite this pull request against the two decisions it puts to a reviewer.

Row 1 is marked in progress rather than complete: the spec says the
table is updated as each pull request *lands*, and marking one's own
unmerged work complete in its own diff is not a claim this programme
should be making.  It flips on merge.

Decision 6 -- whether the moved names drop their underscore prefix --
gains the one piece of evidence implementation actually produced.  The
unprefixed `array_id` collides with eight existing local bindings, five
of which would raise `UnboundLocalError` if left alone.  Every rename in
this pull request beyond the seven definitions exists to clear that
name's path, so declining the rename is not merely cheap, it retires all
eight and leaves a pure `git mv`.

Co-Authored-By: Claude Opus 5 <[email protected]>
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (greenfield@c0dc1d8). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff              @@
##             greenfield    #7276   +/-   ##
=============================================
  Coverage              ?   90.36%           
=============================================
  Files                 ?       94           
  Lines                 ?    25822           
  Branches              ?     4796           
=============================================
  Hits                  ?    23333           
  Misses                ?     1709           
  Partials              ?      780           

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bjlittle

Copy link
Copy Markdown
Member Author

Codex review (model: GPT-5): I reviewed the full change against greenfield, including the hashing relocation, all updated call sites, the renamed hashing tests, the runtime-import invariant, and the design/changelog updates. I found no actionable issues.

The relocated implementation matches the original apart from the documented symbol and local-variable renames. Repository-wide inspection found no remaining callers of the old _concatenate hashing names, git diff --check is clean, and all GitHub CI checks are passing.

On the open naming question, I recommend keeping the unprefixed names. The containing _combine_common module already marks the API as private, while points_id, bounds_id, and key communicate the local roles more clearly than the shadowing names they replace.

Residual risk is low: an external consumer importing these private symbols directly from iris._concatenate would break, but there are no such callers in the repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant