Reference Issues/PRs

Towards: #27931

Follow-up to: #26391 and #23595

What does this implement/fix? Explain your changes.

• Adds missing-value support to RandomSplitter in _splitter.pyx
• Enables the "random" splitter kwarg for ExtraTreeClassifier and ExtraTreeRegressor
• Adds unit-tests for ExtraTreeClassifier and ExtraTreeRegressor
• Make unit-tests for DecisionTree* and ExtraTree* more numerically robust by increasing the tolerance for a GLOBAL_RANDOM_SEED check, and using cross-validation scores rather than a single score

Any other comments?

Compared to BestSplitter, there can be an expected cost to doing splits on missing-values, as we can either:

1. choose a random threshold and then randomly send missing-values to left/right, OR
2. send all missing-values to left/right and all non-missing values to right/left

The push of missing values down the tree can be done randomly (i.e. first option), OR the second option can actually be evaluated. There is a computational cost to doing so, but more importantly there is an interpretation tradeoff. The tradeoff imo comes from the assumption of the missing-values:

• if missing-completely-at-random (MCAR), then option 1 is ideal because one simply should ignore the missing values or impute them
• if missing-at-random (MAR), then option 2 is nice because sometimes the missing-ness of the data can be informative.

However, I think the difference at a tree level is not super important. E.g. in the ExtraTree forest, #28268 demonstrates that the ExtraTrees when combined as a forest are resilient and predictive for missing-values.

Benchmarks demonstrating no significant runtime performance degradation

There is some complexity involved in checking if there are missing values. However, this only occurs at the Python level as shown by the following benchmark. In terms of the Cython code, there is no regression.

Also ran this benchmark for ExtraTrees, which demonstrates that this check is negligible at the forest level, since it only occurs once. See #28268, which has the short code to enable it for ExtraTrees.