Tags: IBM/velox
Tags
fix(ci): Disable sparse-checkout cone mode for Claude context files (f… …acebookincubator#17713) Summary: The `CI Failure Comment` workflow has been failing on every run since git 2.54 landed on `ubuntu-latest` runners (around May 20). The `Checkout for Claude context and prompt` step passes file paths to sparse-checkout: ```yaml sparse-checkout: | CLAUDE.md .claude/CLAUDE.md .claude/skills/ci-failure-analysis/SKILL.md ``` `actions/checkout` enables cone mode by default, and git 2.54 hard-rejects non-directory entries in cone mode: ``` fatal: '.claude/CLAUDE.md' is not a directory; to treat it as a directory anyway, rerun with --skip-checks ``` Earlier git versions on the runner (e.g. 2.53.0 on May 15 runs) accepted file paths in cone mode, which is why the same YAML used to work. This change sets `sparse-checkout-cone-mode: false` on the affected step so file paths are accepted. The earlier `.github/scripts` checkout stays in cone mode — it is a directory and works fine. Pull Request resolved: facebookincubator#17713 Reviewed By: srsuryadev Differential Revision: D107456949 Pulled By: kgpai fbshipit-source-id: 62aded58697523d51a0cd9ddaea501d3327a3fb1
Fix iceberg min max statistics for decimal type when encoded as int32 Signed-off-by: Hazmi <[email protected]> Alchemy-item: (ID = 1203) Fix iceberg min max statistics for decimal type when encoded as int32 commit 1/1 - 0ac9930
[OAP] fix iceberg reader to use split reader Signed-off-by: Yuan <[email protected]> Signed-off-by: Mohammad Linjawi <[email protected]> (cherry picked from commit 4cd49ab) (cherry picked from commit 4fc58be) Alchemy-item: (ID = 1537) [OAP] fix to use spliter iceberg reader commit 1/1 - 064cc0e
feat: Support casting from large negative-exponent string to decimal (f… …acebookincubator#17665) Summary: The PR adds support for casting from large negative-exponent string to decimal zero, to align with Spark and Presto. E.g., the PR adds support for `CAST('6E-120' AS DECIMAL(38, 0)) == 0`, instead of returning an error. This is the step 2 in facebookincubator#17593, following the previous fix facebookincubator#17594. Pull Request resolved: facebookincubator#17665 Reviewed By: apurva-meta Differential Revision: D107421146 Pulled By: bikramSingh91 fbshipit-source-id: 11dc31328bed487036abb00f75f9c4fd3b281e76
Fix iceberg min max statistics for decimal type when encoded as int32 Signed-off-by: Hazmi <[email protected]> Alchemy-item: (ID = 1203) Fix iceberg min max statistics for decimal type when encoded as int32 commit 1/1 - 0ac9930
[OAP] fix iceberg reader to use split reader Signed-off-by: Yuan <[email protected]> Signed-off-by: Mohammad Linjawi <[email protected]> (cherry picked from commit 4cd49ab) (cherry picked from commit 4fc58be) Alchemy-item: (ID = 1537) [OAP] fix to use spliter iceberg reader commit 1/1 - 064cc0e
feat(cudf): Add Iceberg support to velox-cudf (facebookincubator#17054) Summary: ## Description This PR implements GPU-accelerated Iceberg support in Velox-cuDF. Non-exhaustive list of implemented features: - Adds a new set of `CudfIcebergXX` classes derived from `CudfHiveXX` bases. Same class setup as in `velox/connectors/hive` - Similar to the base `CudfHiveConnector`, `CudfIcebergConnector` only supports Parquet data files. - Delete files may be in any format supported by the Velox `RowReader`s. - The workhorse class: `CudfIcebergSplitReader` derives from the `CudfSplitReader` and adds deletion and schema evolution (contributed by ducndh) support to the read `cudf::table`s. - GPU accelerated roaring bitmaps are applied using `cudf::roaring_bitmap` (V3). - Uses Velox's `PositionalDeleteFileReader` to fetch delete positions (V2). - Adds a new `CudfEqualityDeleteFileReader` for GPU accelerated equality delete support (V2). - Lots and lots of new tests. - Reading files using multiple splits using the `splitOffset` field is not yet supported. ## Checklist - [x] All velox-cudf tests are passing - [x] I am familiar with the contributing guide Pull Request resolved: facebookincubator#17054 Reviewed By: kevinwilfong Differential Revision: D106419556 Pulled By: kKPulla fbshipit-source-id: 32c06cd381c3ffd59e01f47c9933e24350ab46f5
perf: Remove TraceContext from DWIO I/O hot paths (facebookincubator#… …17693) Summary: Pull Request resolved: facebookincubator#17693 CONTEXT: Strobelight profiling shows `TraceContext::TraceContext` consuming significant CPU on the I/O read path. Each constructor call allocates a `std::string` label, reads `steady_clock::now()`, pushes to `TraceHistory`, and updates the thread-local registry — all on every batch read, column load, and I/O operation. WHAT: Remove all `process::TraceContext` usage from the DWIO I/O hot paths: - `ColumnLoader::loadInternal` (called per lazy column load) - `TransformColumnLoader::loadInternal` - `DeltaUpdateColumnLoader::loadInternal` - `SelectiveStructColumnReaderBase::next` (called per batch) - `DirectInputStream::loadSync` (called per I/O read) - `CacheInputStream::loadSync` (called per cache load) - `CachedBufferedInput` and `DirectBufferedInput` read-ahead lambdas - `CacheInputStream` error-path `TraceContext::statusLine()` call Reviewed By: tanjialiang Differential Revision: D107201661 fbshipit-source-id: 2ab58bb26c2616aef95e04745920ba92c45b8b5e
Fix iceberg min max statistics for decimal type when encoded as int32 Signed-off-by: Hazmi <[email protected]> Alchemy-item: (ID = 1203) Fix iceberg min max statistics for decimal type when encoded as int32 commit 1/1 - 0ac9930
Fix iceberg min max statistics for decimal type when encoded as int32 Signed-off-by: Hazmi <[email protected]> Alchemy-item: (ID = 1203) Fix iceberg min max statistics for decimal type when encoded as int32 commit 1/1 - 0ac9930
PreviousNext