Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Complete Fetch Phase (EXTERNAL_LINKS disposition and ARROW format) #598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 298 commits into
base: sea-migration
Choose a base branch
from

Conversation

varun-edachali-dbx
Copy link
Collaborator

@varun-edachali-dbx varun-edachali-dbx commented Jun 16, 2025

What type of PR is this?

  • Feature

Description

Complete the Fetch phase of the SEA implementation for EXTERNAL_LINKS disposition and ARROW_STREAM format. Introduce a new CloudFetchQueue.

Note that this is not yet feature complete, with some known failures in the following domains (for the SEA backend only):

  • retries
  • native, complex params
  • metadata queries

These will be resolved in future PRs.

How is this tested?

  • Unit tests
  • E2E Tests
  • Manually - using the test scripts invoked in examples/experimental/sea_connector_test.py.
  • N/A

The coverage of the key classes by the unit tests are as below:

Module Statements Missing Coverage Notes
backend/sea/queue.py (SeaCloudFetchQueue and SeaResultSetQueueFactory classes) 92 2 98% Missing lines are pyarrow import exception handling (lines 10-11) which are not meant to be covered in runtime tests
backend/sea/result_set.py (SeaResultSet class) 85 3 96% Missing lines are pyarrow import exception handling (lines 13-14) and TYPE_CHECKING import (line 17) which are not meant to be covered in runtime tests

Related Tickets & Documents

Design Doc
PECOBLR-553

Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Copy link

github-actions bot commented Jul 4, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@databricks databricks deleted a comment from github-actions bot Jul 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant