perf: add a chunk cache to avoid decoding duplicated miniblock chunks #4846
Conversation
|
ACTION NEEDED The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. For details on the error please inspect the "PR Title Check" action. |
| // Now we iterate through each instruction and process it | ||
| for (instructions, chunk) in self.instructions.iter() { | ||
| // TODO: It's very possible that we have duplicate `buf` in self.instructions and we | ||
| // don't want to decode the buf again and again on the same thread. |
There was a problem hiding this comment.
This PR partially addresses this TODO. It improves performance unless chunks are accessed in a fully random pattern, which would require a HashMap-based cache at the cost of higher memory usage.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4846 +/- ##
=======================================
Coverage 80.84% 80.84%
=======================================
Files 330 330
Lines 130545 130561 +16
Branches 130545 130561 +16
=======================================
+ Hits 105539 105552 +13
- Misses 21269 21272 +3
Partials 3737 3737
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
f1a8fd7 to
611a69c
Compare
…us chunks don't have to be decoded multiple times.
611a69c to
b7620b1
Compare
Description
When miniblock encoding is used in a Lance file, reading the file with the v2
FileReadervia theread_stream_projectedAPI can become inefficient if the providedReadBatchParams::Indicescontains many nearby but non-contiguous row indices.For example:
This kind of access pattern causes the same chunk to be decoded repeatedly, resulting in slow performance and high CPU usage.
Solution
This PR introduces a lightweight single-entry cache in
DecodePageTask. While it only helps when chunks are accessed in a somewhat sequential manner, row indices are typically sorted in ascending order, so the cache strikes a balance between saving memory and improving performance.Test
On a local setup with a Lance file containing 100k rows (each row with a text column of 200+ bytes):
zstdis used for general compression