@seberg Could you give me an example of what you'd like to see happen here?

The way spin spawns the actual spawns the test process, lldb doesn't wrap it. Not sure if there is a clear solution though (maybe I need to tell lldb to wrap the child process or just launch the tests more explicitly).

Basically, add a test like:

def test_crash():
    import numpy as np
    from numpy.lib.stride_tricks import as_strided
    a = as_strided(np.arange(10), (10_000_000,), (8,))
    a[...] = 3

which segfaults. If you just run it with spin lldb -- python -mspin test -t ... you should find that it's not running inside lldb and doesn't stop for debugging. (Even disabling the fault handler with PYTEST_ADDOPTS="-p no:faulthandler").

Maybe there is a very simple work-around. I think last time I around I made a minimal example and used spin lldb -c "exec(open('test.py').read())" (to both run in the right process and to be sure to find the right numpy install).

I looked into this a bit, but I don't know lldb well enough to proceed.

I tried -O settings set target.process.stop-on-exec false, I tried to change spin test to replace the current process, etc. and I could not get lldb to see the traceback.

With the first option I get the traceback, but trying something like bt it says the process has aborted.

Are you able to help me explore the solutions here?

Happy to poke this a bit, but not sure what to actually try. I can't find a good hint on how one might tell lldb to follow the new process better.
I.e. settings set target.process.stop-on-exec false or target.process.follow-fork-mode (parent|child) has no effect? (This thread makes me think maybe it just can't work on MacOS, but I am not sure it is the same problem.)

One thing to help if you poke more might be running with:

PYTEST_ADDOPTS=" -p no:faulthandler"

That will disable the Python traceback from the faulthandler. But I don't recall it should change behavior, but better safe than sorry.

I tried to change spin test to replace the current process, etc. and I could not get lldb to see the traceback.

That was the solution I was hoping might work relatively easily. If it doesn't than maybe it just isn't to be (until someone more knowledgeable teaches us a new trick...).

It's possible that some of those solutions work; it's hard for me to tell, since I don't know what "successful" behavior is. Do you have an example I can look at that does not involve spin, that shows how a traceback should present?

Ah sorry, the "success" is that you get a segfault and can do a bt at that point.

To see it, put the "test" into a file, test.py. NOTE: I increased the fake array size here, without that it wasn't reliably and would crash very cryptically!

def test_crash():
    import numpy as np
    from numpy.lib.stride_tricks import as_strided
    a = as_strided(np.arange(10), (1_000_000_000,), (8,))
    a[1_000_000_000-1] = 3

test_crash()

then run:

spin lldb -c "exec(open('test.py').read())"

and of course then r to actually run. It should crash at least almost always with a traceback, something like:\

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x6001dd495138)
    frame #0: 0x0000000100de3540 _multiarray_umath.cpython-311-darwin.so`LONG_copyswap [inlined] _basic_copy(dst=0x00006001dd495138, src=0x000000016fdfc710, elsize=8) at arraytypes.c.src:1931:5 [opt]
   1928	    if (src == NULL) {
   1929	        return;
   1930	    }
-> 1931	    memcpy(dst, src, elsize);
   1932	}
   1933	
   1934	
Target 0: (python3.11) stopped.

And bt should will sho something obvious. (If bt does anything at all, things are probably fine!)

OTOH, I suppose there is a decent chance the bug is in mypy, and I guess then we would need to sanitize mypy to get anywhere...

We chatted about this on the NumPy triage meeting today and I shared that spin lldb does still work for me if I do something like:

spin lldb -- $(pyenv which python) $(pyenv which pytest) "build-install/usr/lib/python3.13/site-packages/numpy/_core/tests/test_stringdtype.py"

For this to work, you need to pass the absolute path to the python intepreter, the pytest script, and the relative path to the build-install version of the test you want to debug. In particular it doesn't work with e.g. python -m pytest (at least for NumPy) because that has slightly different test discovery semantics that are broken.

Maybe spin could help make this less arcane in the UI?

Ouch, yes, we certainly should!

It would be easy enough to launch the correct python and pytest, but not sure how we'd deal with the build-dir. I presume this doesn't work?

spin lldb -- $(pyenv which python) $(pyenv which pytest) numpy/_core/tests/test_stringdtype.py

If it did, we could add, e.g., spin lldb -t numpy/_core_tests_test_stringdtype.py.

Nope, for whatever reason, that hits an error:

± spin lldb -- $(pyenv which python) $(pyenv which pytest) numpy/_core/tests/test_stringdtype.py
python(94040,0x20c141f00) malloc: nano zone abandoned due to inability to reserve vm space.
Invoking `build` prior to invoking lldb:
$ /Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python vendored-meson/meson/meson.py compile -C build
python(94094,0x20c141f00) malloc: nano zone abandoned due to inability to reserve vm space.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/ninja -C /Users/goldbaum/Documents/numpy/build
ninja: Entering directory `/Users/goldbaum/Documents/numpy/build'
[1/1] Generating numpy/generate-version with a custom command
Saving version to numpy/version.py
$ /Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python vendored-meson/meson/meson.py install --only-changed -C build --destdir ../build-install
$ export PYTHONPATH="/Users/goldbaum/Documents/numpy/build-install/usr/lib/python3.13/site-packages"
$ lldb -O 'settings set target.process.follow-fork-mode child' -- /Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python /Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/pytest numpy/_core/tests/test_stringdtype.py
(lldb) settings set target.process.follow-fork-mode child
(lldb) target create "/Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python"
Current executable set to '/Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python' (arm64).
(lldb) settings set -- target.run-args  "/Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/pytest" "numpy/_core/tests/test_stringdtype.py"
(lldb) r
Process 94122 launched: '/Users/goldbaum/.pyenv/versions/3.13-dev-asan/bin/python' (arm64)
python(94122,0x20c141f00) malloc: nano zone abandoned due to inability to reserve vm space.
ImportError while loading conftest '/Users/goldbaum/Documents/numpy/numpy/conftest.py'.
numpy/__init__.py:93: in <module>
    from . import version
E   ImportError: cannot import name 'version' from partially initialized module 'numpy' (most likely due to a circular import) (/Users/goldbaum/Documents/numpy/numpy/__init__.py)
Process 94122 exited with status = 4 (0x00000004)

Right, so we'll need the same workaround we use for our regular test command to avoid importing from the wrong path.

In this case, I wonder if perhaps we add a flag to the test command, so that we wrap lldb around it?

It's very much a half-baked PR, but take a look and let's see if we can iterate on that until it's working: #281