Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Scan mixed line endings without line vector#2160

Merged
j178 merged 1 commit into
masterfrom
perf-mixed-line-ending-scan
Jun 6, 2026
Merged

Scan mixed line endings without line vector#2160
j178 merged 1 commit into
masterfrom
perf-mixed-line-ending-scan

Conversation

@j178

@j178 j178 commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • count line endings directly from the file bytes instead of first collecting line slices
  • rewrite line endings with a direct byte scan, avoiding an intermediate line vector

Tests

  • cargo test -p prek --bin prek hooks::pre_commit_hooks::mixed_line_ending::tests
  • PREK_INTERNAL__TEST_DIR=D:/Projects/Rust/prek/target/prek-tests cargo test -p prek --test builtin_hooks mixed_line_ending_hook
  • git -c safe.directory=D:/Projects/Rust/prek diff --check

@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.44444% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.58%. Comparing base (d30253b) to head (2f8fced).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
...ek/src/hooks/pre_commit_hooks/mixed_line_ending.rs 94.44% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2160      +/-   ##
==========================================
- Coverage   92.59%   92.58%   -0.01%     
==========================================
  Files         122      122              
  Lines       26168    26152      -16     
==========================================
- Hits        24231    24214      -17     
- Misses       1937     1938       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@prek-ci-bot

prek-ci-bot Bot commented Jun 6, 2026

Copy link
Copy Markdown

📦 Cargo Bloat Comparison

Binary size change: +0.38% (26.2 MiB → 26.3 MiB)

Expand for cargo-bloat output

Head Branch Results

 File  .text     Size             Crate Name
 1.2%   2.6% 332.0KiB        aws_lc_sys aws_lc_0_41_0_aes_gcm_encrypt_avx512
 1.2%   2.6% 332.0KiB        aws_lc_sys aws_lc_0_41_0_aes_gcm_decrypt_avx512
 0.3%   0.7%  86.7KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.7%  86.2KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.6%  78.4KiB             prek? <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.3%   0.5%  69.8KiB              prek prek::languages::<impl prek::config::Language>::install::{{closure}}
 0.2%   0.4%  49.4KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.4%  48.9KiB              prek prek::run::{{closure}}
 0.2%   0.3%  44.6KiB              prek prek::cli::run::run::run::{{closure}}
 0.1%   0.2%  31.6KiB             prek? <prek::cli::RunOptions as clap_builder::derive::Args>::augment_args
 0.1%   0.2%  30.6KiB             prek? <prek::config::_::<impl serde_core::de::Deserialize for prek::config::Config>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2%  30.1KiB               std core::ptr::drop_in_place<prek::languages::<impl prek::config::Language>::install::{{closure}}>
 0.1%   0.2%  28.1KiB      serde_saphyr granit_parser::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2%  28.0KiB        aws_lc_sys aws_lc_0_41_0_edwards25519_scalarmuldouble_alt
 0.1%   0.2%  27.5KiB        aws_lc_sys aws_lc_0_41_0_edwards25519_scalarmuldouble
 0.1%   0.2%  26.3KiB              prek prek::cli::try_repo::try_repo::{{closure}}
 0.1%   0.2%  23.0KiB      serde_saphyr granit_parser::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2%  22.3KiB         [Unknown] Lp384_montjscalarmul_alt_p384_montjadd
 0.1%   0.2%  21.8KiB     serde_saphyr? <serde_saphyr::de::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map
 0.1%   0.2%  21.5KiB      clap_builder clap_builder::parser::parser::Parser::get_matches_with
41.7%  86.2%  10.9MiB                   And 24119 smaller methods. Use -n N to show more.
48.3% 100.0%  12.7MiB                   .text section size, the file size is 26.3MiB

Base Branch Results

 File  .text     Size             Crate Name
 1.2%   2.6% 332.0KiB        aws_lc_sys aws_lc_0_41_0_aes_gcm_encrypt_avx512
 1.2%   2.6% 332.0KiB        aws_lc_sys aws_lc_0_41_0_aes_gcm_decrypt_avx512
 0.3%   0.7%  86.7KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.7%  86.2KiB              prek prek::languages::<impl prek::config::Language>::run::{{closure}}::{{closure}}
 0.3%   0.5%  69.8KiB              prek prek::languages::<impl prek::config::Language>::install::{{closure}}
 0.2%   0.5%  65.2KiB             prek? <prek::cli::Command as clap_builder::derive::Subcommand>::augment_subcommands
 0.2%   0.4%  49.4KiB annotate_snippets annotate_snippets::renderer::render::render
 0.2%   0.4%  48.9KiB              prek prek::run::{{closure}}
 0.2%   0.3%  44.6KiB              prek prek::cli::run::run::run::{{closure}}
 0.1%   0.2%  31.9KiB             prek? <prek::cli::RunOptions as clap_builder::derive::Args>::augment_args
 0.1%   0.2%  30.6KiB             prek? <prek::config::_::<impl serde_core::de::Deserialize for prek::config::Config>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map
 0.1%   0.2%  30.1KiB               std core::ptr::drop_in_place<prek::languages::<impl prek::config::Language>::install::{{closure}}>
 0.1%   0.2%  28.1KiB      serde_saphyr granit_parser::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2%  28.0KiB        aws_lc_sys aws_lc_0_41_0_edwards25519_scalarmuldouble_alt
 0.1%   0.2%  27.5KiB        aws_lc_sys aws_lc_0_41_0_edwards25519_scalarmuldouble
 0.1%   0.2%  26.3KiB              prek prek::cli::try_repo::try_repo::{{closure}}
 0.1%   0.2%  23.0KiB      serde_saphyr granit_parser::scanner::Scanner<T>::fetch_more_tokens
 0.1%   0.2%  22.3KiB         [Unknown] Lp384_montjscalarmul_alt_p384_montjadd
 0.1%   0.2%  21.8KiB     serde_saphyr? <serde_saphyr::de::YamlDeserializer as serde_core::de::Deserializer>::deserialize_map
 0.1%   0.2%  21.5KiB      clap_builder clap_builder::parser::parser::Parser::get_matches_with
41.7%  86.3%  11.0MiB                   And 24090 smaller methods. Use -n N to show more.
48.3% 100.0%  12.7MiB                   .text section size, the file size is 26.2MiB

@prek-ci-bot

prek-ci-bot Bot commented Jun 6, 2026

Copy link
Copy Markdown

⚡️ Hyperfine Benchmarks

Summary: 0 regressions, 1 improvements above the 10% threshold.

Environment
  • OS: Linux 6.17.0-1015-azure
  • CPU: 4 cores
  • prek version: prek 0.4.4+8 (95f64f5 2026-06-06)
  • Rust version: rustc 1.96.0 (ac68faa20 2026-05-25)
  • Hyperfine version: hyperfine 1.20.0
CLI Commands

Benchmarking basic commands in the main repo:

prek --version

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base --version 2.3 ± 0.1 2.1 2.6 1.07 ± 0.06
prek-head --version 2.2 ± 0.1 2.1 2.5 1.00

prek list

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base list 9.5 ± 0.2 9.2 10.0 1.00
prek-head list 9.6 ± 0.2 9.2 10.3 1.01 ± 0.03

prek validate-config .pre-commit-config.yaml

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base validate-config .pre-commit-config.yaml 3.1 ± 0.1 3.0 3.7 1.01 ± 0.05
prek-head validate-config .pre-commit-config.yaml 3.1 ± 0.1 2.9 3.2 1.00

prek sample-config

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base sample-config 2.5 ± 0.1 2.4 2.8 1.00 ± 0.05
prek-head sample-config 2.5 ± 0.1 2.4 2.8 1.00
Cold vs Warm Runs

Comparing first run (cold) vs subsequent runs (warm cache):

prek run --all-files (cold - no cache)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run --all-files 82.4 ± 3.3 77.0 86.8 1.02 ± 0.05
prek-head run --all-files 80.9 ± 1.8 78.5 83.1 1.00

prek run --all-files (warm - with cache)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run --all-files 79.0 ± 1.5 76.6 82.7 1.00
prek-head run --all-files 81.3 ± 3.8 76.3 88.0 1.03 ± 0.05
Full Hook Suite

Running the builtin hook suite on the benchmark workspace:

prek run --all-files (full builtin hook suite)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run --all-files 80.1 ± 3.1 74.9 87.3 1.00
prek-head run --all-files 80.5 ± 3.5 74.3 92.3 1.01 ± 0.06
Individual Hook Performance

Benchmarking each hook individually on the test repo:

prek run trailing-whitespace --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run trailing-whitespace --all-files 22.0 ± 0.6 21.1 23.3 1.02 ± 0.03
prek-head run trailing-whitespace --all-files 21.4 ± 0.3 20.9 22.0 1.00

prek run end-of-file-fixer --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run end-of-file-fixer --all-files 27.5 ± 1.7 25.2 31.4 1.00
prek-head run end-of-file-fixer --all-files 27.7 ± 1.9 24.7 31.1 1.01 ± 0.09

prek run check-json --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-json --all-files 8.4 ± 0.3 7.9 9.1 1.00
prek-head run check-json --all-files 8.5 ± 0.2 8.0 9.1 1.01 ± 0.05

prek run check-yaml --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-yaml --all-files 8.1 ± 0.2 7.7 8.5 1.00
prek-head run check-yaml --all-files 8.1 ± 0.2 7.7 8.5 1.00 ± 0.03

prek run check-toml --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-toml --all-files 8.5 ± 0.3 7.9 9.2 1.01 ± 0.06
prek-head run check-toml --all-files 8.4 ± 0.3 7.8 9.2 1.00

prek run check-xml --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-xml --all-files 8.3 ± 0.2 7.8 8.6 1.00
prek-head run check-xml --all-files 8.6 ± 0.3 8.1 9.2 1.05 ± 0.05

prek run detect-private-key --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run detect-private-key --all-files 14.9 ± 1.6 12.7 18.4 1.01 ± 0.14
prek-head run detect-private-key --all-files 14.7 ± 1.2 12.7 17.6 1.00

prek run fix-byte-order-marker --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run fix-byte-order-marker --all-files 23.3 ± 1.6 20.7 26.5 1.00
prek-head run fix-byte-order-marker --all-files 23.4 ± 2.0 20.4 28.1 1.00 ± 0.11
Installation Performance

Benchmarking hook installation (fast path hooks skip Python setup):

prek install-hooks (cold - no cache)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base install-hooks 4.9 ± 0.2 4.8 5.2 1.04 ± 0.04
prek-head install-hooks 4.7 ± 0.1 4.7 4.8 1.00

prek install-hooks (warm - with cache)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base install-hooks 4.9 ± 0.1 4.7 5.0 1.02 ± 0.04
prek-head install-hooks 4.8 ± 0.1 4.6 5.0 1.00
File Filtering/Scoping Performance

Testing different file selection modes:

prek run (staged files only)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run 40.1 ± 3.6 38.0 53.4 1.00 ± 0.10
prek-head run 40.1 ± 1.9 38.2 44.7 1.00

prek run --files '*.json' (specific file type)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run --files '*.json' 28.7 ± 33.6 8.4 108.8 3.34 ± 3.90
prek-head run --files '*.json' 8.6 ± 0.2 8.2 9.1 1.00

✅ Performance improvement for prek run --files '*.json' (specific file type): 70.0400% faster

Workspace Discovery & Initialization

Benchmarking hook discovery and initialization overhead:

prek run --dry-run --all-files (measures init overhead)

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run --dry-run --all-files 7.1 ± 0.1 6.9 7.4 1.00
prek-head run --dry-run --all-files 7.2 ± 0.1 7.0 7.6 1.01 ± 0.02
Meta Hooks Performance

Benchmarking meta hooks separately:

prek run check-hooks-apply --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-hooks-apply --all-files 12.4 ± 0.2 12.0 12.7 1.00
prek-head run check-hooks-apply --all-files 12.5 ± 0.2 12.2 13.0 1.01 ± 0.03

prek run check-useless-excludes --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run check-useless-excludes --all-files 12.5 ± 0.6 11.3 13.2 1.11 ± 0.06
prek-head run check-useless-excludes --all-files 11.3 ± 0.3 10.9 11.9 1.00

prek run identity --all-files

Command Mean [ms] Min [ms] Max [ms] Relative
prek-base run identity --all-files 10.5 ± 0.1 10.3 10.8 1.01 ± 0.02
prek-head run identity --all-files 10.4 ± 0.1 10.2 10.5 1.00

@j178 j178 added the performance Performance improvements label Jun 6, 2026
@j178 j178 marked this pull request as ready for review June 6, 2026 09:17
Copilot AI review requested due to automatic review settings June 6, 2026 09:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the mixed line ending pre-commit hook to avoid building an intermediate “lines” vector by scanning the raw file bytes directly for both detection and rewriting, improving efficiency while keeping behavior consistent.

Changes:

  • Reworked line-ending counting to scan bytes in-place (handling CRLF as a single unit).
  • Reworked line-ending rewriting to perform a single-pass byte scan and rebuild output without collecting line slices.

@j178 j178 merged commit f8a2a28 into master Jun 6, 2026
35 checks passed
@j178 j178 deleted the perf-mixed-line-ending-scan branch June 6, 2026 09:56
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Jun 17, 2026
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [prek](https://github.com/j178/prek) | patch | `0.4.4` → `0.4.5` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>j178/prek (prek)</summary>

### [`v0.4.5`](https://github.com/j178/prek/blob/HEAD/CHANGELOG.md#045)

[Compare Source](j178/prek@v0.4.4...v0.4.5)

Released on 2026-06-15.

##### ✨Highlights

0.4.5 completes `prek`'s pre-commit language coverage. With R, Conda, Perl,
and coursier now supported, `prek` supports every language available in
pre-commit, plus `prek`-only modern languages such as Bun and Deno.

This is full feature parity with pre-commit. The last major compatibility gap
is closed, and `prek` is now on the path to 1.0.

##### 💚 Sponsorship

If `prek` saves time for you or your team, please consider sponsoring the
project on [GitHub Sponsors](https://github.com/sponsors/j178). It helps keep
new features, performance work, and maintenance moving.

##### Enhancements

- Add `Perl` language support ([#&#8203;2201](j178/prek#2201))
- Add `R` language support ([#&#8203;2206](j178/prek#2206))
- Add `Conda` language support ([#&#8203;2203](j178/prek#2203))
- Add `coursier` language support ([#&#8203;2194](j178/prek#2194))
- Add group filters to list ([#&#8203;2152](j178/prek#2152))
- Remove legacy repo path support ([#&#8203;2150](j178/prek#2150))
- Skip cloning fully filtered remote repos ([#&#8203;2151](j178/prek#2151))

##### Performance

- Avoid building JSON trees in syntax hooks ([#&#8203;2156](j178/prek#2156))
- Scan mixed line endings without line vector ([#&#8203;2160](j178/prek#2160))
- Skip file scan for always-run no-filename hooks ([#&#8203;2155](j178/prek#2155))
- Use fixed buffer for EOF scan ([#&#8203;2163](j178/prek#2163))
- Use fixed buffer for private key scan ([#&#8203;2164](j178/prek#2164))

##### Bug fixes

- Fix pretty-format-json CRLF handling ([#&#8203;2210](j178/prek#2210))
- Handle missing blobs in diff snapshots ([#&#8203;2211](j178/prek#2211))
- Ignore coursier channel from working repo ([#&#8203;2202](j178/prek#2202))

##### Contributors

- [@&#8203;j178](https://github.com/j178)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTguMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6cGF0Y2giXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants