ENH add aligned numpy allocator #29846
Conversation
|
I recently tried to debug a segfault / bus error that only happened on the CI using an ad-hoc PR with specific instrumentation here: #29628 This PR is a bit specific to macOS but it can probably be adapted to debug segfaults on linux. I wonder if it would be worth the extra maintenance complexity to have something similar enabled by default on our CI. |
Do you observe any performance impact? |
|
@ogrisel The segfault also occurs for linux, see CI results. (For Windows, this PR needs a slight update with C macros). I need this PR to work in order to test the impact on performance. |
|
Do you need help with debugging? |
Yes. I have no clue how to debug that segfault. But note that I don’t know if it’s worthwhile. |
|
If performance and (not just) memory alignment are of interest, we could try and mention using |
Reference Issues/PRs
None
What does this implement/fix? Explain your changes.
This PR adds a numpy allocator for aligned memory in order to evaluate the performance impact (mainly on HGBT).
LightGBM uses aligned allocation, see https://github.com/microsoft/LightGBM/blob/41ba9e8f00c89d72e5cb71c964722ce1ed4d8445/include/LightGBM/utils/common.h#L909.
Numpy allocator C-API, see
Any other comments?
While numpy/numpy#27327 works perfectly fine locally, this PR results in a segmentation fault.