Codestin Search App

neutrinoceros · 2024-07-15T13:38:36Z

Description

This is a work in progress.
Rationale:

the lack of support for np.einsum for MaskedNDArray is currently blocking ENH: include exclusive values from IERS_B in IERS_Auto (blocked by #16725) #16070 and @mhvk has expressed interest in supporting it
the function is currently marked as "TODO" in a comment, alongside a bunch of other product-like functions that we could all get almost for free if we just get this one right.

Now, clearly, I didn't know what I was getting myself into when I proposed doing this, but I'm making progress. I'm opening this draft PR at a relatively early stage so it doesn't get lost in my constant context-switching dance.
At the time of writing, only the last test is failing. Fixing this is my next priority.

By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

github-actions · 2024-07-15T13:39:03Z

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

Do the proposed changes actually accomplish desired goals?
Do the proposed changes follow the Astropy coding guidelines?
Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

github-actions · 2024-07-15T13:39:06Z

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

mhvk

Some initial comments, mostly to suggest an easier way to calculate the final mask...

mhvk · 2024-07-23T16:41:57Z

+            raise NotImplementedError
+
+        else:
+            kwargs_with_out["out"] = out.unmasked.view(np.ndarray)


Why the .view(np.ndarray)? If this is, say, a Quantity, then I think it should still be dispatched on that.

mhvk · 2024-07-23T16:44:18Z

+            mask = np.full_like(input, fill_value=False, dtype=bool)
+            return Masked(input, mask=mask)
+
+    mas = tuple(as_masked_ndarray(op) for op in operands)


I think you want arrays, masks = _get_data_and_masks(*operands)?

mhvk · 2024-07-23T16:47:37Z

+
+    masks = tuple(ma.mask for ma in mas)
+
+    from collections import Counter


Are you sure this whole sequence is necessary? If we stick with that any masked element propagates correctly, then can we not just use the same trick as for matmul in __array_ufunc__:

astropy/astropy/utils/masked/core.py

Lines 780 to 794 in 0ea3682

# np.matmul is tricky and its signature cannot be parsed by

# _parse_gufunc_signature. But we can calculate the mask

# with matmul by using that nan will propagate correctly.

# We use float16 to minimize the memory requirements.

nan_masks = []

for a, m in zip(unmasked, masks):

nan_mask = np.zeros(a.shape, dtype=np.float16)

if m is not None:

nan_mask[m] = np.nan

nan_masks.append(nan_mask)

m_kwargs = {

k: v for k, v in kwargs.items() if k not in ("out", "where")

}

t = ufunc(*nan_masks, **m_kwargs)

mask = np.isnan(t, out=out_mask)

(Note that it did take me quite a while to realize this might work...)

mhvk · 2024-07-23T16:48:53Z

+        result = retv
+
+    # TODO: understand what the third item is supposed to be
+    return result, mask, None


Third item is a possible out parameter (which should be Masked)

neutrinoceros · 2024-07-29T13:49:22Z

Thanks for this first round @mhvk, and especially for suggesting a simpler impl. I was initially happy with my current strategy, but the last failing test made me think that maybe I actually painted myself into a corner.
Anyway, I just wanted to let you know that I'm taking August off and I don't think I'll be able to get back to you just before that, so I'll give your idea a try in September !

mhvk · 2024-07-29T14:16:26Z

Bonnes vacances!

neutrinoceros · 2024-10-16T07:51:53Z

I don't think I'll have time to return to this before the feature freeze so let's move it to 7.1

github-actions · 2024-12-13T05:42:11Z

Hi humans 👋 - this pull request hasn't had any new commits for approximately 4 months. I plan to close this in 30 days if the pull request doesn't have any new commits by then.

In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add keep-open label to keep this PR open but it is discouraged unless absolutely necessary.

If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock.

If you believe I commented on this pull request incorrectly, please report this here.

neutrinoceros · 2024-12-16T10:45:00Z

Still planning to come back to this eventually... let me rebase the branch and disable autoclosing

github-actions · 2025-05-16T05:40:18Z

Hi humans 👋 - this pull request hasn't had any new commits for approximately 4 months. I plan to close this in 30 days if the pull request doesn't have any new commits by then.

In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add keep-open label to keep this PR open but it is discouraged unless absolutely necessary.

If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock.

If you believe I commented on this pull request incorrectly, please report this here.

github-actions Bot added the utils.masked label Jul 15, 2024

pllim added this to the v7.0.0 milestone Jul 15, 2024

neutrinoceros changed the title ~~ENH: implement np.einsum for MaskedNDArray~~ ENH: implement np.einsum for MaskedNDArray Jul 15, 2024

mhvk reviewed Jul 23, 2024

View reviewed changes

neutrinoceros modified the milestones: v7.0.0, v7.1.0 Oct 16, 2024

github-actions Bot added the Close? Tell stale bot that this issue/PR is stale label Dec 13, 2024

ENH: implement np.einsum for MaskedNDArray

10f1539

neutrinoceros force-pushed the utils.masked/implement_np.einsum branch from 5d2cd53 to 10f1539 Compare December 16, 2024 10:45

neutrinoceros removed the Close? Tell stale bot that this issue/PR is stale label Dec 16, 2024

astrofrog modified the milestones: v7.1.0, v7.2.0 Apr 28, 2025

github-actions Bot added the Close? Tell stale bot that this issue/PR is stale label May 16, 2025

neutrinoceros added keep-open and removed Close? Tell stale bot that this issue/PR is stale labels May 16, 2025

astrofrog modified the milestones: v7.2.0, v8.0.0 Nov 3, 2025

astrofrog modified the milestones: v8.0.0, v8.1.0 May 5, 2026


		masks = tuple(ma.mask for ma in mas)

		from collections import Counter

	# np.matmul is tricky and its signature cannot be parsed by
	# _parse_gufunc_signature. But we can calculate the mask
	# with matmul by using that nan will propagate correctly.
	# We use float16 to minimize the memory requirements.
	nan_masks = []
	for a, m in zip(unmasked, masks):
	nan_mask = np.zeros(a.shape, dtype=np.float16)
	if m is not None:
	nan_mask[m] = np.nan
	nan_masks.append(nan_mask)
	m_kwargs = {
	k: v for k, v in kwargs.items() if k not in ("out", "where")
	}
	t = ufunc(nan_masks, *m_kwargs)
	mask = np.isnan(t, out=out_mask)

Uh oh!

Uh oh!

Conversation

neutrinoceros commented Jul 15, 2024

Description

Uh oh!

github-actions Bot commented Jul 15, 2024

Uh oh!

github-actions Bot commented Jul 15, 2024

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

mhvk Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

mhvk Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

mhvk Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

mhvk Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

neutrinoceros commented Jul 29, 2024

Uh oh!

mhvk commented Jul 29, 2024

Uh oh!

neutrinoceros commented Oct 16, 2024

Uh oh!

github-actions Bot commented Dec 13, 2024

Uh oh!

neutrinoceros commented Dec 16, 2024

Uh oh!

github-actions Bot commented May 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants