Codestin Search App

prady0t · 2026-05-15T07:17:44Z

Reference issue

What does this implement/fix?

This is a draft PR to check how much we have progressed with xp_assert functions in the array_api_extra library. We are checking how easy it is for other libraries to implement these testing functions in their existing test suite.

Additional information

This is to remain a draft PR for reference. In case we want to merge it, I can put some effort into cleaning it and making some necessary changes in the test_array_api.py file. There are 2 things different in the array_api_extra version of tests from scipy:

Checking namespaces cannot be disabled, which can be done in SciPy. See
SciPy allows for Python lists and scalars in the assert functions; these are not allowed in the array_api_extra version. See

I've made some efforts to take these into account and reduce the number of failing tests by checking for Python scalars and lists, and converting them with xp.asarray before feeding them to array_api_extra.
Most test failures are due to our inability to disable namespace checks; they originate from test_array_api.py.

See:

AI Generation Disclosure

No AI tools used

Signed-off-by: Pradyot Ranjan <[email protected]>

prady0t · 2026-05-15T17:13:15Z

We have 8 total failures. All of them are expected namespace mismatch failures, as array_api_extra doesn't allow disabling the namespace check.

Signed-off-by: Pradyot Ranjan <[email protected]>

Co-authored-by: Lucas Colley <[email protected]>

Signed-off-by: Pradyot Ranjan <[email protected]>

lucascolley · 2026-05-16T17:54:49Z

The GPU failures demonstrate something we need to address — atol and rtol should either be explicitly rejected if they are arrays (rather than Python scalars), or explicitly accepted and converted either to NumPy arrays or a scalar type. I'm not sure exactly what we'd want yet.

The 'install into venv' failure I think can be addressed when we make the assertions public by ensuring that their module does not depend on pytest.

RuntimeError: Can't call numpy() on Tensor that has conjugate bit set. Use tensor.resolve_conj().numpy() instead.

That's probably something we want to fix over in array-api-extra.

lucascolley · 2026-05-16T18:08:39Z

The GPU failures demonstrate something we need to address — atol and rtol should either be explicitly rejected if they are arrays (rather than Python scalars), or explicitly accepted and converted either to NumPy arrays or a scalar type. I'm not sure exactly what we'd want yet.

I think we should try explicitly accepting 0-D arrays in array-api-extra for these parameters, but rejecting higher-dimensional arrays. If there are a bunch of failures for 1-D arrays with a single element then maybe it makes sense to accept that also.

prady0t · 2026-05-16T18:34:19Z

or explicitly accepted and converted either to NumPy arrays or a scalar type.

I don't know much about pytorch tensors, but can we do

float(4e-16 * ref.integral)

In the tests to make this work?

Or : https://docs.pytorch.org/docs/2.12/tensors.html

prady0t · 2026-05-16T18:38:17Z

But ofc this won't be a proper fix as we need to address these inputs in the extra module.

lucascolley · 2026-05-17T10:16:46Z

Taking a step back, the main feature of assertions is that they are stable and "boring", so that when porting a library or writing a new library, the author can concentrate on the functionality and rely on assertions just doing the right thing. What exactly is the right thing can be open to discussions of course, and we had way too many of those in scipy at least (I'm sure elsewhere too), which is why I thought they just work, and there's no need to rewrite then stabilize them, again.

Yeah, part of the motivation for avoiding use of native assertions is for this kind of stability (or at least consistency) — we don't want to have to deal with bugs or nuances of various different xp.testing libraries if we can achieve it all via np.testing anyway.

I think you have a good argument if the question was "should we put in a bunch of thought to re-write these now?" But really, that work has already been done by Guido and Matt, so I think we are right to make use of it (presuming, fairly, that that work is of good quality).

Apparently not without changes; unfortunate but well, okay.

Let's check what changes would be required before we finalise the public API! Would you be willing to try these out in scikit-image and report any difficulties? I already have my eyes on wilsonrljr/sysidentpy#232 and glass-dev/glass#1076 too.

default_xp: this one is large and extremely useful.

I think this may be the key remaining area of work.

Signed-off-by: Pradyot Ranjan <[email protected]>

prady0t · 2026-05-17T19:51:11Z

strictness and knobs to turn some checks off. Strictness, accepting lists vs array-likes vs arrays only, check_* keywords etc --- this is something a library can and will customize; what's needed downstream is a way to customize the behavior without copy-pasting the whole thing, with just small wrappers on top of xpx assertions.

default_xp: this one is large and extremely useful.

xp_capabilities should probably best decoupled from core assertions.

I think these are useful points. Numpy also has a strict parameter; if set to True, it checks for dtypes, scalars, and shape. It could be useful for libraries that want to rely on this parameter. Can we open a tracking issue in array_api_extra for these improvements?

lucascolley · 2026-05-17T20:00:56Z

Numpy also has a strict parameter; if set to True, it checks for dtypes, scalars, and shape. It could be useful for libraries that want to rely on this parameter.

I would wait until it is explicitly requested, the finer grained parameters we have already are better IMO.

prady0t · 2026-05-17T20:03:50Z

the finer grained parameters we have already are better IMO.

True.

prady0t · 2026-05-18T03:45:26Z

All checks are passing now!

lucascolley · 2026-05-18T08:26:22Z

nice, I think #25143 (comment) is the next step

Signed-off-by: Pradyot Ranjan <[email protected]>

prady0t · 2026-05-19T21:12:54Z

Only assert_array_almost_equal_nulp remains as a non-array_api_extra testing utility here. Can we port it too? If so, we can remove _strict_check and _assert_matching_namespace helper functions entirely from here.

lucascolley · 2026-05-19T21:14:32Z

Only assert_array_almost_equal_nulp remains as a non-array_api_extra testing utility here. Can we port it too? If so, we can remove _strict_check and _assert_matching_namespace helper functions entirely from here.

Let's leave it for now, but yes let's return to it when we bring this PR out of draft.

[skip circle]

Co-authored-by: Lucas Colley <[email protected]>

lucascolley · 2026-05-25T14:50:50Z

I think data-apis/array-api-extra#755 is the next step here

lucascolley · 2026-05-26T23:07:57Z

+
+
+def xp_assert_equal(actual, desired, *, check_dtype=True,
+                    check_shape=True, check_0d=False, err_msg='', xp=None):


Suggested change

check_shape=True, check_0d=False, err_msg='', xp=None):

check_shape=True, check_0d=True, err_msg='', xp=None):

Actually, I kept it False on purpose.
Before scipy did

actual = xp.asarray(actual) desired = xp.asarray(desired)

To all arrays, but we now do:

if isinstance(x, (list, tuple)) or type(x) in ( int, float, complex, bool, ): return xp.asarray(x)

(only use asarrya for Python scalar and lists.)

This leads to many a place where a scalar input becomes a 0D array, and hence check_0d fails (where it was being compared to numpy scalars).
In scipy, for this reason, we first check_scalar before converting to xp.asarray, see

scipy/scipy/_lib/_array_api.py

Line 238 in 9f253f9

# only NumPy distinguishes between scalars and arrays; we do if check_0d=True.

. We can't do that in array_api_extra.

If we want to keep it True, I can give check_0d=False as parameters for tests that currently fail with this change.

This leads to many a place where a scalar input becomes a 0D array, and hence check_0d fails (where it was being compared to numpy scalars).

can you give a concrete example? I thought the type(x) in (int, float, complex, bool) would fail for NumPy scalars.

It doesn't actually. Only works for Python scalars.See here for example:

scipy/scipy/signal/tests/test_waveforms.py

Line 55 in 9f253f9

assert_almost_equal(w, 1.0)

w here is an np scalar, this raises

E AssertionError: array-ness does not match: E Actual: <class 'numpy.float64'> E Desired: <class 'numpy.ndarray'>

are there cases where we hit this via the xp_... assertions, or only those with the same names as the ones from np.testing?

This line also produces a similar error:

scipy/scipy/signal/tests/test_waveforms.py

Line 113 in 9f253f9

xp_assert_close(w, 1.0+0j)

E AssertionError: array-ness does not match: E Actual: <class 'numpy.complex128'> E Desired: <class 'numpy.ndarray'>

thanks, I pushed a commit which aims to address this.

Nicely done 👏🏼

Co-authored-by: Lucas Colley <[email protected]>

[skip circle]

lucascolley · 2026-05-28T14:55:03Z

time to use assert_close_nulp 👀?

Signed-off-by: Pradyot Ranjan <[email protected]>

prady0t added 2 commits May 9, 2026 16:48

initial testing of extra's private testing functions

f7409d7

Signed-off-by: Pradyot Ranjan <[email protected]>

Experimental usage of array_api_extra's private testing functions

160f238

Signed-off-by: Pradyot Ranjan <[email protected]>

github-actions Bot added scipy.interpolate scipy.cluster scipy._lib array types Items related to array API support and input array validation (see gh-18286) labels May 15, 2026

prady0t changed the title ~~Use array api extra testing~~ Use array_api_extra's testing functions May 15, 2026

prady0t mentioned this pull request May 15, 2026

Experimental usage of array_api_extra's private testing functions scikit-learn/scikit-learn#34019

Draft

4 tasks

Changing defaut value of check_0d to False

34fab3b

Signed-off-by: Pradyot Ranjan <[email protected]>

lucascolley reviewed May 15, 2026

View reviewed changes

Comment thread scipy/interpolate/tests/test_bsplines.py Outdated

Comment thread scipy/_lib/_array_api.py Outdated

removing extra check_shape input

4f03581

Signed-off-by: Pradyot Ranjan <[email protected]>

lucascolley reviewed May 16, 2026

View reviewed changes

Comment thread scipy/interpolate/tests/test_bsplines.py Outdated

Comment thread scipy/_lib/_array_api.py Outdated

Comment thread scipy/_lib/_array_api.py Outdated

Comment thread scipy/_lib/_array_api.py Outdated

lucascolley mentioned this pull request May 16, 2026

ENH: add testing support utilities data-apis/array-api-extra#17

Open

lucascolley reviewed May 16, 2026

View reviewed changes

Comment thread scipy/_lib/tests/test_array_api.py Outdated

prady0t and others added 2 commits May 16, 2026 17:43

Update scipy/interpolate/tests/test_bsplines.py

88a6f6b

Co-authored-by: Lucas Colley <[email protected]>

adding scalar check function and fixing tests

d92e52f

Signed-off-by: Pradyot Ranjan <[email protected]>

lucascolley mentioned this pull request May 16, 2026

backend-agnostic Compare glass-dev/glass#1076

Open

lucascolley reviewed May 16, 2026

View reviewed changes

Comment thread scipy/_lib/_array_api.py Outdated

lucascolley mentioned this pull request May 16, 2026

generic interface to np.testing assertions wilsonrljr/sysidentpy#232

Open

lucascolley changed the title ~~Use array_api_extra's testing functions~~ TST: use xpx.testing assertions May 16, 2026

lucascolley added maintenance Items related to regular maintenance tasks and removed scipy.interpolate scipy.cluster scipy._lib labels May 16, 2026

removing check_namespace usages

8ed4c0f

Signed-off-by: Pradyot Ranjan <[email protected]>

prady0t mentioned this pull request May 17, 2026

BUG: testing: convert torch tensors with conjugate bit set to NumPy data-apis/array-api-extra#744

Merged

lucascolley added this to the 1.19.0 milestone May 17, 2026

prady0t added 2 commits May 17, 2026 22:35

using xpx_assert_less function and pointing to latest extra

3640fc5

Signed-off-by: Pradyot Ranjan <[email protected]>

lint failures + dask broadcast failures

2dfeb4c

Signed-off-by: Pradyot Ranjan <[email protected]>

paddyroddy mentioned this pull request May 19, 2026

gh-1076: Use array comparison functions from array-api-extra glass-dev/glass#1083

Draft

5 tasks

ev-br mentioned this pull request May 19, 2026

POC: Array API compatibility scikit-image/scikit-image#8182

Open

addressing new changes in extra

3574062

Signed-off-by: Pradyot Ranjan <[email protected]>

keep default_xp here

4e0a864

[skip circle]

lucascolley force-pushed the use_array-api-extra-testing branch from 92416b6 to 4e0a864 Compare May 22, 2026 15:26

use public API

2cacd02

lucascolley reviewed May 24, 2026

View reviewed changes

Comment thread scipy/_lib/_array_api.py Outdated

Update scipy/_lib/_array_api.py

4f5d0d7

Co-authored-by: Lucas Colley <[email protected]>

lucascolley reviewed May 24, 2026

View reviewed changes

Comment thread scipy/_lib/_array_api.py Outdated

Update scipy/_lib/_array_api.py

f67e666

Co-authored-by: Lucas Colley <[email protected]>

lucascolley reviewed May 26, 2026

View reviewed changes

prady0t and others added 3 commits May 28, 2026 12:48

Update scipy/_lib/_array_api.py

c48b7dd

Co-authored-by: Lucas Colley <[email protected]>

Update scipy/_lib/_array_api.py

11af99c

Co-authored-by: Lucas Colley <[email protected]>

TST: use no_0d assertions in some places

bce12e6

[skip circle]

lucascolley force-pushed the use_array-api-extra-testing branch from 5947421 to bce12e6 Compare May 28, 2026 11:22

pointing to latest array_api_extra and using assert_close_nulp

44cd051

Signed-off-by: Pradyot Ranjan <[email protected]>



		def xp_assert_equal(actual, desired, *, check_dtype=True,
		check_shape=True, check_0d=False, err_msg='', xp=None):

	check_shape=True, check_0d=False, err_msg='', xp=None):
	check_shape=True, check_0d=True, err_msg='', xp=None):

Uh oh!

Conversation

prady0t commented May 15, 2026

Reference issue

What does this implement/fix?

Additional information

AI Generation Disclosure

Uh oh!

prady0t commented May 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lucascolley commented May 16, 2026

Uh oh!

lucascolley commented May 16, 2026

Uh oh!

prady0t commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

prady0t commented May 16, 2026

Uh oh!

lucascolley commented May 17, 2026

Uh oh!

prady0t commented May 17, 2026

Uh oh!

lucascolley commented May 17, 2026

Uh oh!

prady0t commented May 17, 2026

Uh oh!

prady0t commented May 18, 2026

Uh oh!

lucascolley commented May 18, 2026

Uh oh!

prady0t commented May 19, 2026

Uh oh!

lucascolley commented May 19, 2026

Uh oh!

Uh oh!

Uh oh!

lucascolley commented May 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lucascolley commented May 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

prady0t commented May 16, 2026 •

edited

Loading