FIX avoid non-inlined isnan() calls in tree/HGB hot loops - #34876

Would defining our own isnan inline function in a well-chosen .pxd work in terms of performance and be a bit more explicit about the intent as well as less verbose? I.e. something like this (with a better name 😉):

cdef inline bint more_likely_inlined_isnan(double x) noexcept:
    return x != x

Maybe there are still some cases where it wouldn't be inlined but chatting with an AI (chAItting 😁) seems to tell me that our own inline Cython function is more likely to be inlined than lib.math.isnan.

That's a very good idea, and very safe for inlining yes.

There is sklearn/utils/_typedefs.pxd that's already imported everywhere. It's not really a typedef but it's a bit a type things, so I'd say it's not too weird to put it there.

How to name it? isnan might collide so maybe not a good idea; is_nan, is_nan_inlined, ...?

isnan is fine. Just add a docstring that explains why it's better to use this than calling the libm library function.

I added isnan in sklearn/utils/_typedefs.pxd and verified it still fixes the slowdown ✅

Apparently, there is a risk that aggressive floating-point optimization flags remove completely x != x, so I switched for bit-mask (no visible performance impact).

I added a test too, it'll fail if one day someone switches to x != x + fast-math flag.

Does using npy_isnan also fix the problem (from numpy cimport npy_isnan)?

If that works, then I guess it's preferable to rely on numpy to do this kind of low-level tricks, rather than rolling our own "solution".

Apparently numpy use the compiler builtin isnan if it exists, and AI is telling me it does at least for gcc and clang (MSVC maybe not).
https://github.com/numpy/numpy/blob/8fecb3b1c054b5eae955b30c334cdf471bdd53ee/numpy/_core/include/numpy/npy_math.h#L212-L218
In particular this comment seems quite relevant for the problem at hand 😉

/* use builtins to avoid function calls in tight loops

My AI says:

I dug into this — the suggestion doesn't actually hold up for scikit-learn. Here's why:

What numpy's npy_isnan does:

#ifdef HAVE___BUILTIN_ISNAN
    #define npy_isnan(x) __builtin_isnan(x)
#else
    #define npy_isnan(x) isnan(x)
#endif

HAVE___BUILTIN_ISNAN only gets defined via numpy's private npy_config.h, which is generated during numpy's own build and is explicitly not shipped in the public headers (numpy/_core/include/numpy/) that downstream packages like scikit-learn compile against. I confirmed this two ways:

1. npy_config.h doesn't exist anywhere under numpy.get_include().
2. Grepped scikit-learn's actual ninja build args for every -DHAVE... flag — none are set.

So for scikit-learn, npy_isnan(x) unconditionally expands to plain isnan(x) — i.e. it's a no-op rename of exactly what we already have via libc.math.isnan. I verified this by preprocessing numpy's header with sklearn's real include paths.

OK thanks for trying, I still have a small preference for having a different name for our custom isnan. This would make it more easily greppable and easier to differentiate vs libm isnan.

Happy to be overriden on this one, if I am the only one who thinks this 😉. I did see above @ogrisel thinks using the isnan name is fine.

Suggestions:

• sklearn_isnan (following a bit the np_isnan name)
• inlinable_isnan
• something better?

We could also copy what numpy does with using compiler builtin isnan maybe?

Fine with either.

I'm putting back this PR as draft:

• for modern compilation chain this bitmask approach is likely a bit slower
• there seem to be some movement on the conda-forge side, they might bump glibc for everyone soon (edit: not soon). I'll keep track of where discussions go there.
• We can also bump it for our recipe only (around here) before the next release.

I chose inlinable_isnan, I like self-explanatory names.

As discussed IRL yesterday, let's do this fix rather than the glibc bump.