Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Speedup push_down_filter_regression.slt by using uncompressed parquet#20652

Open
alamb wants to merge 1 commit intoapache:mainfrom
alamb:alamb/less_zstd
Open

Speedup push_down_filter_regression.slt by using uncompressed parquet#20652
alamb wants to merge 1 commit intoapache:mainfrom
alamb:alamb/less_zstd

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Mar 2, 2026

Which issue does this PR close?

Rationale for this change

push_down_filter_regression.slt is the sqllogictest that takes the longest to run, even after @Tim-53 reduced its time in

While reviewing #20586 and trying to make the sqllogictest runs faster, I noticed that a substantial amount of the unit test time was spent doing zstd compression/decompression:

Screenshot 2026-03-02 at 12 50 24 PM

Thus, we can improve the test speed by skipping the zstd step

What changes are included in this PR?

  1. Don't compress the parquet files in the test

Are these changes tested?

Yes by CI

Here are my performance runs using @kosiew 's new timing feature

cargo test --profile=ci --test sqllogictests  -- --timing-summary top

Main:

Per-file elapsed summary (deterministic):
1.    4.035s  push_down_filter_regression.slt  <-- takes  over 4 seconds
2.    3.573s  joins.slt
3.    3.492s  aggregate.slt
4.    3.316s  imdb.slt
5. ```

This PR

Per-file elapsed summary (deterministic):

  1. 3.308s aggregate.slt
  2. 3.290s joins.slt
  3. 3.181s imdb.slt
  4. 2.914s push_down_filter_regression.slt <--- takes less than 3 seconds and is no longer the tallest pole

## Are there any user-facing changes?

Faster tests

@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Mar 2, 2026
@Dandandan
Copy link
Contributor

Dandandan commented Mar 2, 2026

What about setting opt-level=1 for [profile.ci]?
Got some good results with that as well around 2x improvement while only minimally slower compile times (sometimes it migh be even faster).

@alamb
Copy link
Contributor Author

alamb commented Mar 2, 2026

What about setting opt-level=1 for [profile.ci]? Got some good results with that as well around 2x improvement while only minimally slower compile times (sometimes it migh be even faster).

Definitely worth checking out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speedup execution of sqllogictests with more parallelization

2 participants