Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Introduce preliminary SEA Result Set #588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 68 commits into from
Jun 12, 2025
Merged

Conversation

varun-edachali-dbx
Copy link
Collaborator

@varun-edachali-dbx varun-edachali-dbx commented Jun 9, 2025

What type of PR is this?

  • Feature

Description

  • Preliminary SeaResultSet with no behaviour (only a constructor)
  • Helps to ease the introduction of the execution phase of the SEA client since it must return a SeaResultSet from execution relevant methods.

How is this tested?

  • Unit tests
  • E2E Tests
  • Manually
  • N/A

The coverage of SeaResultSet by the unit tests in test_sea_result_set.py is as below:

Module Statements Missing Coverage Notes
result_set.py (SeaResultSet class) 39 0 100% Fully covered

Related Tickets & Documents

https://docs.google.com/document/d/1Y-eXLhNqqhrMVGnOlG8sdFrCxBTN1GdQvuKG4IfHmo0/edit?usp=sharing

Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
@varun-edachali-dbx varun-edachali-dbx changed the base branch from main to exec-resp-norm June 9, 2025 08:58
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
Signed-off-by: varun-edachali-dbx <[email protected]>
covered by #588 anyway

Signed-off-by: varun-edachali-dbx <[email protected]>
Copy link

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

This reverts commit a70a6ce.

Signed-off-by: varun-edachali-dbx <[email protected]>
Copy link
Contributor

@samikshya-db samikshya-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Signed-off-by: varun-edachali-dbx <[email protected]>
@varun-edachali-dbx varun-edachali-dbx merged commit bb3f15a into sea-migration Jun 12, 2025
22 of 23 checks passed
varun-edachali-dbx added a commit that referenced this pull request Jun 18, 2025
* [squash from exec-sea] bring over execution phase changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess test

Signed-off-by: varun-edachali-dbx <[email protected]>

* add docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remvoe exec func in sea backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess files

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess models

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess sea backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* cleanup

Signed-off-by: varun-edachali-dbx <[email protected]>

* re-introduce get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* clean imports and attributes

Signed-off-by: varun-edachali-dbx <[email protected]>

* pass CommandId to ExecResp

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove changes in types

Signed-off-by: varun-edachali-dbx <[email protected]>

* add back essential types (ExecResponse, from_sea_state)

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix fetch types

Signed-off-by: varun-edachali-dbx <[email protected]>

* excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff by maintaining logs

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix int test types

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from exec-sea] init execution func

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove ResultSetFilter functionality

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* even more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove sea response as init option

Signed-off-by: varun-edachali-dbx <[email protected]>

* exec test example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from sea-exec] merge sea stuffs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess removed docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes in backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidentally removed _get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unnecessary init with sea_response tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* rmeove unnecessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* improved models and filters from cloudfetch-sea branch

Signed-off-by: varun-edachali-dbx <[email protected]>

* filters stuff (align with JDBC)

Signed-off-by: varun-edachali-dbx <[email protected]>

* backend from cloudfetch-sea

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove filtering, metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* raise NotImplementedErrror for metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* change to valid table name

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

covered by #588

Signed-off-by: varun-edachali-dbx <[email protected]>

* simplify test module

Signed-off-by: varun-edachali-dbx <[email protected]>

* logging -> debug level

Signed-off-by: varun-edachali-dbx <[email protected]>

* change table name in log

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary backend cahnges

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

only relevant in Fetch phase

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication in response parsing

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication

Signed-off-by: varun-edachali-dbx <[email protected]>

* more clear docstrings

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce strongly typed ChunkInfo

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove is_volume_operation from response

Signed-off-by: varun-edachali-dbx <[email protected]>

* add is_volume_op and more ResultData fields

Signed-off-by: varun-edachali-dbx <[email protected]>

* add test scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* Revert "Merge branch 'exec-models-sea' into exec-phase-sea"

This reverts commit be1997e, reversing
changes made to 37813ba.

* change logging level

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove _get_schema_bytes (for now)

Signed-off-by: varun-edachali-dbx <[email protected]>

* redundant comments

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove fetch phase methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code repetititon + introduce gaps after multi line pydocs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* move description extraction to helper func

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* add more unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* streamline unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* test getting the list of allowed configurations

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* house constants in enums for readability and immutability

Signed-off-by: varun-edachali-dbx <[email protected]>

* add note on hybrid disposition

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove redundant note on arrow_schema_bytes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove invalid import

Signed-off-by: varun-edachali-dbx <[email protected]>

* add strong typing for manifest in _extract_description

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary column skipping

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove parsing in backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix: convert sea statement id to CommandId type

Signed-off-by: varun-edachali-dbx <[email protected]>

* make polling interval a separate constant

Signed-off-by: varun-edachali-dbx <[email protected]>

* align state checking with Thrift implementation

Signed-off-by: varun-edachali-dbx <[email protected]>

* update unit tests according to changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* add unit tests for added methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* add spec to description extraction docstring, add strong typing to params

Signed-off-by: varun-edachali-dbx <[email protected]>

* add strong typing for backend parameters arg

Signed-off-by: varun-edachali-dbx <[email protected]>

---------

Signed-off-by: varun-edachali-dbx <[email protected]>
varun-edachali-dbx added a commit that referenced this pull request Jun 26, 2025
* [squash from exec-sea] bring over execution phase changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess test

Signed-off-by: varun-edachali-dbx <[email protected]>

* add docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remvoe exec func in sea backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess files

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess models

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess sea backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* cleanup

Signed-off-by: varun-edachali-dbx <[email protected]>

* re-introduce get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* clean imports and attributes

Signed-off-by: varun-edachali-dbx <[email protected]>

* pass CommandId to ExecResp

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove changes in types

Signed-off-by: varun-edachali-dbx <[email protected]>

* add back essential types (ExecResponse, from_sea_state)

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix fetch types

Signed-off-by: varun-edachali-dbx <[email protected]>

* excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff by maintaining logs

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix int test types

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from exec-sea] init execution func

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove ResultSetFilter functionality

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* even more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove sea response as init option

Signed-off-by: varun-edachali-dbx <[email protected]>

* exec test example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from sea-exec] merge sea stuffs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess removed docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes in backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidentally removed _get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unnecessary init with sea_response tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* rmeove unnecessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* improved models and filters from cloudfetch-sea branch

Signed-off-by: varun-edachali-dbx <[email protected]>

* filters stuff (align with JDBC)

Signed-off-by: varun-edachali-dbx <[email protected]>

* backend from cloudfetch-sea

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove filtering, metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* raise NotImplementedErrror for metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* add metadata commands

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* add metadata command unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* change to valid table name

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

covered by #588

Signed-off-by: varun-edachali-dbx <[email protected]>

* simplify test module

Signed-off-by: varun-edachali-dbx <[email protected]>

* logging -> debug level

Signed-off-by: varun-edachali-dbx <[email protected]>

* change table name in log

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary backend cahnges

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

only relevant in Fetch phase

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication in response parsing

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication

Signed-off-by: varun-edachali-dbx <[email protected]>

* more clear docstrings

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce strongly typed ChunkInfo

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove is_volume_operation from response

Signed-off-by: varun-edachali-dbx <[email protected]>

* add is_volume_op and more ResultData fields

Signed-off-by: varun-edachali-dbx <[email protected]>

* add test scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* Revert "Merge branch 'sea-migration' into exec-models-sea"

This reverts commit 8bd12d8, reversing
changes made to 030edf8.

* Revert "Merge branch 'exec-models-sea' into exec-phase-sea"

This reverts commit be1997e, reversing
changes made to 37813ba.

* change logging level

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove _get_schema_bytes (for now)

Signed-off-by: varun-edachali-dbx <[email protected]>

* redundant comments

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove fetch phase methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code repetititon + introduce gaps after multi line pydocs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* move description extraction to helper func

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* add more unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* streamline unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* test getting the list of allowed configurations

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* house constants in enums for readability and immutability

Signed-off-by: varun-edachali-dbx <[email protected]>

* add note on hybrid disposition

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove redundant note on arrow_schema_bytes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary test changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes in thrift backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented methods test

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove invalid import

Signed-off-by: varun-edachali-dbx <[email protected]>

* better align queries with JDBC impl

Signed-off-by: varun-edachali-dbx <[email protected]>

* line breaks after multi-line PRs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix: introduce ExecuteResponse import

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented metadata methods test, un-necessary imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce unit tests for metadata methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove verbosity in ResultSetFilter docstring

Co-authored-by: jayant <[email protected]>

* remove un-necessary info in ResultSetFilter docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove explicit type checking, string literals around forward annotations

Signed-off-by: varun-edachali-dbx <[email protected]>

* house SQL commands in constants

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove catalog requirement in get_tables

Signed-off-by: varun-edachali-dbx <[email protected]>

* move filters.py to SEA utils

Signed-off-by: varun-edachali-dbx <[email protected]>

* ensure SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* prevent circular imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove cast, throw error if not SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* make SEA backend methods return SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* use spec-aligned Exceptions in SEA backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove defensive row type check

Signed-off-by: varun-edachali-dbx <[email protected]>

---------

Signed-off-by: varun-edachali-dbx <[email protected]>
Co-authored-by: jayant <[email protected]>
varun-edachali-dbx added a commit that referenced this pull request Jun 27, 2025
* [squash from exec-sea] bring over execution phase changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess test

Signed-off-by: varun-edachali-dbx <[email protected]>

* add docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remvoe exec func in sea backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess files

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess models

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess sea backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* cleanup

Signed-off-by: varun-edachali-dbx <[email protected]>

* re-introduce get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* clean imports and attributes

Signed-off-by: varun-edachali-dbx <[email protected]>

* pass CommandId to ExecResp

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove changes in types

Signed-off-by: varun-edachali-dbx <[email protected]>

* add back essential types (ExecResponse, from_sea_state)

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix fetch types

Signed-off-by: varun-edachali-dbx <[email protected]>

* excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff by maintaining logs

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix int test types

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from exec-sea] init execution func

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove ResultSetFilter functionality

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* even more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove sea response as init option

Signed-off-by: varun-edachali-dbx <[email protected]>

* exec test example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* change to valid table name

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

covered by #588

Signed-off-by: varun-edachali-dbx <[email protected]>

* simplify test module

Signed-off-by: varun-edachali-dbx <[email protected]>

* logging -> debug level

Signed-off-by: varun-edachali-dbx <[email protected]>

* change table name in log

Signed-off-by: varun-edachali-dbx <[email protected]>

* add basic documentation on env vars to be set

Signed-off-by: varun-edachali-dbx <[email protected]>

---------

Signed-off-by: varun-edachali-dbx <[email protected]>
varun-edachali-dbx added a commit that referenced this pull request Jul 2, 2025
…t) (#594)

* [squash from exec-sea] bring over execution phase changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess test

Signed-off-by: varun-edachali-dbx <[email protected]>

* add docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remvoe exec func in sea backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess files

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess models

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess sea backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* cleanup

Signed-off-by: varun-edachali-dbx <[email protected]>

* re-introduce get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* clean imports and attributes

Signed-off-by: varun-edachali-dbx <[email protected]>

* pass CommandId to ExecResp

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove changes in types

Signed-off-by: varun-edachali-dbx <[email protected]>

* add back essential types (ExecResponse, from_sea_state)

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix fetch types

Signed-off-by: varun-edachali-dbx <[email protected]>

* excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff by maintaining logs

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix int test types

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from exec-sea] init execution func

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove ResultSetFilter functionality

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* even more irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove sea response as init option

Signed-off-by: varun-edachali-dbx <[email protected]>

* exec test example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* [squashed from sea-exec] merge sea stuffs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess removed docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes in backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidentally removed _get_schema_desc

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unnecessary init with sea_response tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* rmeove unnecessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* move guid_to_hex_id import to utils

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff in guid utils import

Signed-off-by: varun-edachali-dbx <[email protected]>

* improved models and filters from cloudfetch-sea branch

Signed-off-by: varun-edachali-dbx <[email protected]>

* move arrow_schema_bytes back into ExecuteResult

Signed-off-by: varun-edachali-dbx <[email protected]>

* maintain log

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary assignment

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary tuple response

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-ncessary verbose mocking

Signed-off-by: varun-edachali-dbx <[email protected]>

* filters stuff (align with JDBC)

Signed-off-by: varun-edachali-dbx <[email protected]>

* move Queue construction to ResultSert

Signed-off-by: varun-edachali-dbx <[email protected]>

* move description to List[Tuple]

Signed-off-by: varun-edachali-dbx <[email protected]>

* frmatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff (remove explicit tuple conversion)

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove has_more_rows from ExecuteResponse

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary has_more_rows aclc

Signed-off-by: varun-edachali-dbx <[email protected]>

* default has_more_rows to True

Signed-off-by: varun-edachali-dbx <[email protected]>

* return has_more_rows from ExecResponse conversion during GetRespMetadata

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unnecessary replacement

Signed-off-by: varun-edachali-dbx <[email protected]>

* better mocked backend naming

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove has_more_rows test in ExecuteResponse

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce replacement of original has_more_rows read test

Signed-off-by: varun-edachali-dbx <[email protected]>

* call correct method in test_use_arrow_schema

Signed-off-by: varun-edachali-dbx <[email protected]>

* call correct method in test_fall_back_to_hive_schema

Signed-off-by: varun-edachali-dbx <[email protected]>

* re-introduce result response read test

Signed-off-by: varun-edachali-dbx <[email protected]>

* simplify test

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess fetch_results mocks

Signed-off-by: varun-edachali-dbx <[email protected]>

* more minimal changes to thrift_backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* move back to old table types

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove outdated arrow_schema_bytes return

Signed-off-by: varun-edachali-dbx <[email protected]>

* backend from cloudfetch-sea

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove filtering, metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* raise NotImplementedErrror for metadata ops

Signed-off-by: varun-edachali-dbx <[email protected]>

* align SeaResultSet with new structure

Signed-off-by: varun-edachali-dbx <[email protected]>

* correct sea res set tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* add metadata commands

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* add metadata command unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* minimal fetch phase intro

Signed-off-by: varun-edachali-dbx <[email protected]>

* working JSON + INLINE

Signed-off-by: varun-edachali-dbx <[email protected]>

* change to valid table name

Signed-off-by: varun-edachali-dbx <[email protected]>

* rmeove redundant queue init

Signed-off-by: varun-edachali-dbx <[email protected]>

* large query results

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

covered by #588

Signed-off-by: varun-edachali-dbx <[email protected]>

* simplify test module

Signed-off-by: varun-edachali-dbx <[email protected]>

* logging -> debug level

Signed-off-by: varun-edachali-dbx <[email protected]>

* change table name in log

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary backend cahnges

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-needed GetChunksResponse

only relevant in Fetch phase

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication in response parsing

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code duplication

Signed-off-by: varun-edachali-dbx <[email protected]>

* more clear docstrings

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce strongly typed ChunkInfo

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove is_volume_operation from response

Signed-off-by: varun-edachali-dbx <[email protected]>

* add is_volume_op and more ResultData fields

Signed-off-by: varun-edachali-dbx <[email protected]>

* add test scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* Revert "Merge branch 'sea-migration' into exec-models-sea"

This reverts commit 8bd12d8, reversing
changes made to 030edf8.

* Revert "Merge branch 'exec-models-sea' into exec-phase-sea"

This reverts commit be1997e, reversing
changes made to 37813ba.

* change logging level

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove _get_schema_bytes (for now)

Signed-off-by: varun-edachali-dbx <[email protected]>

* redundant comments

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove fetch phase methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce code repetititon + introduce gaps after multi line pydocs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* move description extraction to helper func

Signed-off-by: varun-edachali-dbx <[email protected]>

* formatting (black)

Signed-off-by: varun-edachali-dbx <[email protected]>

* add more unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* streamline unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* test getting the list of allowed configurations

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff

Signed-off-by: varun-edachali-dbx <[email protected]>

* house constants in enums for readability and immutability

Signed-off-by: varun-edachali-dbx <[email protected]>

* add note on hybrid disposition

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove redundant note on arrow_schema_bytes

Signed-off-by: varun-edachali-dbx <[email protected]>

* align SeaResultSet with ext-links-sea

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove redundant methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* update unit tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidental venv changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* add fetchmany_arrow and fetchall_arrow

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidental changes in sea backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove irrelevant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary test changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes in thrift backend tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented methods test

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented method tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* modify example scripts to include fetch calls

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix sea connector tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented methods test

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove invalid import

Signed-off-by: varun-edachali-dbx <[email protected]>

* better align queries with JDBC impl

Signed-off-by: varun-edachali-dbx <[email protected]>

* line breaks after multi-line PRs

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix: introduce ExecuteResponse import

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unimplemented metadata methods test, un-necessary imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce unit tests for metadata methods

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove verbosity in ResultSetFilter docstring

Co-authored-by: jayant <[email protected]>

* remove un-necessary info in ResultSetFilter docstring

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove explicit type checking, string literals around forward annotations

Signed-off-by: varun-edachali-dbx <[email protected]>

* house SQL commands in constants

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce unit tests for altered functionality

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* run small queries with SEA during integration tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* run some tests for sea

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove catalog requirement in get_tables

Signed-off-by: varun-edachali-dbx <[email protected]>

* move filters.py to SEA utils

Signed-off-by: varun-edachali-dbx <[email protected]>

* ensure SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* prevent circular imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused imports

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove cast, throw error if not SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* pass param as TSparkParameterValue

Signed-off-by: varun-edachali-dbx <[email protected]>

* make SEA backend methods return SeaResultSet

Signed-off-by: varun-edachali-dbx <[email protected]>

* use spec-aligned Exceptions in SEA backend

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove defensive row type check

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce type conversion for primitive types for JSON + INLINE

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SEA running on metadata queries (known failures

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary docstrings

Signed-off-by: varun-edachali-dbx <[email protected]>

* align expected types with databricks sdk

Signed-off-by: varun-edachali-dbx <[email protected]>

* link rest api reference to validate types

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove test_catalogs_returns_arrow_table test

metadata commands not expected to pass

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix fetchall_arrow and fetchmany_arrow

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove thrift aligned test_cancel_during_execute from SEA tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary changes in example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary chagnes in example scripts

Signed-off-by: varun-edachali-dbx <[email protected]>

* _convert_json_table -> _create_json_table

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove accidentally removed test

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove new unit tests (to be re-added based on new arch)

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove changes in sea_result_set functionality (to be re-added)

Signed-off-by: varun-edachali-dbx <[email protected]>

* introduce more integration tests

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove SEA tests in parameterized queries

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove partial parameter fix changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove un-necessary timestamp tests

(pass with minor disparity)

Signed-off-by: varun-edachali-dbx <[email protected]>

* slightly stronger typing of _convert_json_types

Signed-off-by: varun-edachali-dbx <[email protected]>

* stronger typing of json utility func s

Signed-off-by: varun-edachali-dbx <[email protected]>

* stronger typing of fetch*_json

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove unused helper methods in SqlType

Signed-off-by: varun-edachali-dbx <[email protected]>

* line breaks after multi line pydocs, remove excess logs

Signed-off-by: varun-edachali-dbx <[email protected]>

* line breaks after multi line pydocs, reduce diff of redundant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* reduce diff of redundant changes

Signed-off-by: varun-edachali-dbx <[email protected]>

* mandate ResultData in SeaResultSet constructor

Signed-off-by: varun-edachali-dbx <[email protected]>

* return empty JsonQueue in case of empty response

test ref: test_create_table_will_return_empty_result_set

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove string literals around SeaDatabricksClient declaration

Signed-off-by: varun-edachali-dbx <[email protected]>

* move conversion module into dedicated utils

Signed-off-by: varun-edachali-dbx <[email protected]>

* clean up _convert_decimal, introduce scale and precision as kwargs

Signed-off-by: varun-edachali-dbx <[email protected]>

* use stronger typing in convert_value (object instead of Any)

Signed-off-by: varun-edachali-dbx <[email protected]>

* make Manifest mandatory

Signed-off-by: varun-edachali-dbx <[email protected]>

* mandatory Manifest, clean up statement_id typing

Signed-off-by: varun-edachali-dbx <[email protected]>

* stronger typing for fetch*_json

Signed-off-by: varun-edachali-dbx <[email protected]>

* make description non Optional, correct docstring, optimize col conversion

Signed-off-by: varun-edachali-dbx <[email protected]>

* fix type issues

Signed-off-by: varun-edachali-dbx <[email protected]>

* make description mandatory, not Optional

Signed-off-by: varun-edachali-dbx <[email protected]>

* n_valid_rows -> num_rows

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove excess print statement

Signed-off-by: varun-edachali-dbx <[email protected]>

* remove empty bytes in SeaResultSet for arrow_schema_bytes

Signed-off-by: varun-edachali-dbx <[email protected]>

* move SeaResultSetQueueFactory and JsonQueue into separate SEA module

Signed-off-by: varun-edachali-dbx <[email protected]>

* move sea result set into backend/sea package

Signed-off-by: varun-edachali-dbx <[email protected]>

* improve docstrings

Signed-off-by: varun-edachali-dbx <[email protected]>

* correct docstrings, ProgrammingError -> ValueError

Signed-off-by: varun-edachali-dbx <[email protected]>

* let type of rows by List[List[str]] for clarity

Signed-off-by: varun-edachali-dbx <[email protected]>

* select Queue based on format in manifest

Signed-off-by: varun-edachali-dbx <[email protected]>

* make manifest mandatory

Signed-off-by: varun-edachali-dbx <[email protected]>

* stronger type checking in JSON helper functions in Sea Result Set

Signed-off-by: varun-edachali-dbx <[email protected]>

* assign empty array to data array if None

Signed-off-by: varun-edachali-dbx <[email protected]>

* stronger typing in JsonQueue

Signed-off-by: varun-edachali-dbx <[email protected]>

---------

Signed-off-by: varun-edachali-dbx <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants