PERF Vectorize sparse path of PolynomialCountSketch.transform - #34919
PERF Vectorize sparse path of PolynomialCountSketch.transform#34919regarmukesh3g wants to merge 2 commits into
Conversation
|
Thank you for opening your first pull request to scikit-learn! 🎉 To help get your contribution reviewed, please make sure that:
|
f162579 to
3286e80
Compare
The sparse branch of PolynomialCountSketch.transform looped over every feature and every degree, slicing a single sparse column and calling .toarray() on it in each iteration. This densified one (n_samples,) column at a time regardless of the matrix's actual sparsity, so the cost scaled with n_features * degree rather than the number of nonzeros. For a matrix with 1% density and a few thousand features, this made the sparse path an order of magnitude slower than simply densifying the whole input up front and using the dense code path.
0d857c0 to
f1a2db9
Compare
The prior comment re-explained each line of the new sparse branch (one comment per statement, almost matching the code line-for-line), which is exactly the kind of comment well-named identifiers should replace instead. Renamed weighted/projection to signed_features/ bucket_projection and collapsed the comment to a single one above the if-block stating only the non-obvious part: why this avoids per-column densification, which is the actual bug being fixed.
3bf4fc6 to
4d9ec90
Compare
|
Reviewed this independently (I'd arrived at the same sparse-matmul approach before seeing this PR, see #34920). Checked out the branch and ran a 200-trial randomized differential test against the original loop implementation as ground truth (varying shape, degree, small |
Reference Issues/PRs
See #34920 (not using a closing keyword since the issue is still pending
triage).
What does this implement/fix? Explain your changes.
PolynomialCountSketch.transform's sparse branch loops over every featureand every degree, slicing one sparse column at a time and calling
.toarray()on it:This densifies a full
(n_samples,)array on every iteration regardless ofcolumn sparsity, so cost scales with
n_features * degreeinstead of thenumber of nonzeros. On a 1%-density matrix with a few thousand features, the
sparse path ends up an order of magnitude slower than just densifying the
whole input and using the dense branch.
This PR replaces the double loop with one sparse matrix multiplication per
degree: scale columns by
bitHash_[d, :]via.multiply(...), thenscatter-sum into buckets with a sparse 0/1 projection matrix built from
indexHash_[d, :]. Output is unchanged.Local benchmark (2000 samples, 300 components, degree=2, 1% density):
Important: this PR always improves on the code it replaces. The table
below (1000 samples, 200 components, degree=2, 2000 features, varying
density) makes both comparisons explicit — old-vs-new (the one that matters
for approval) and new-vs-dense (context on the sparse/dense crossover):
The old sparse path was uniformly ~0.15-0.16s regardless of density,
because its cost was driven by loop count (
n_features * degree) ratherthan the data itself. The new path's cost scales with the number of
nonzeros, so it is dramatically faster than the old path at every density
tested — 5x to 78x depending on density — even though at high density (>~20%)
it is still somewhat slower than simply densifying the input and using the
unrelated dense branch, which is expected of any sparse algorithm and not a
regression: a 50-80%-dense matrix stored as a sparse type is already an
unusual choice on the caller's part, and the fix never makes that case worse
than main, only better.
Note on dense input: the
elsebranch handling dense arrays is untouched bythis PR (byte-for-byte identical diff), and measured dense
transformtimingis unchanged before/after within run-to-run noise.
Since this is a performance fix rather than a correctness bug, the existing
test_polynomial_count_sketch_dense_sparsetest passes on both the old andnew implementation (both are correct, just different speeds), so it alone
doesn't prove the new code is right. I added
test_polynomial_count_sketch_sparse_edge_cases, covering all-zero sparsecolumns and a single-feature/single-sample input, shapes not exercised by the
existing random test data. I confirmed this new test is not vacuous: I
temporarily broke the new vectorized code (dropped the sign-hash multiply)
and the new test failed with a clear mismatch, then restored the fix.
Verification:
pytest sklearn/tests/test_kernel_approximation.py-> 80 passedcheck_estimator(PolynomialCountSketch())passesruff check/ruff format --checkcleanIntroduce yourself
I use scikit-learn for general ML work. While checking the open-issue
backlog for something to contribute, I found it heavily claimed (most open
Bug/Documentation/good first issue/help wantedissues already had acompeting PR or an active claim comment), so I looked for an improvement by
reading source directly instead, and found this inefficiency in
kernel_approximation.py.AI usage disclosure
I used AI assistance for:
Any other comments?
Already added the
efficiencychangelog entry underdoc/whats_new/upcoming_changes/sklearn.kernel_approximation/34919.efficiency.rst.Opened #34920 to report the bug ahead of this PR per the contributing
guidelines. It's pending triage, so this PR references it without a closing
keyword for now and I'll switch to
Fixes #34920once the label is cleared.