Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

aholyoke
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

According to BQ docs, the syntax for except and intersect is EXCEPT DISTINCT and INTERSECT DISTINCT respectively.

For example,

import sqlalchemy as sa

engine = sa.create_engine("bigquery://")
conn = engine.connect()

meta = sa.MetaData()
table = sa.Table(
    "set_example",
    meta,
    sa.Column("id", sa.Integer()),
    sa.Column("value", sa.Integer()),
    schema=DATASET_NAME,
)
table.create(bind=conn)

conn.execute(table.insert().values([
    {"id": 1, "value": 5},
    {"id": 2, "value": 2},
    {"id": 3, "value": 4},
    {"id": 4, "value": 9},
    {"id": 5, "value": 7},
    {"id": 6, "value": 1},
]))

expr = sa.select(table.c.id).except_(
    sa.select(table.c.id).where(table.c.value >= 5),
)
conn.execute(expr).fetchall()

Before this change we get:

DatabaseError: (google.cloud.bigquery.dbapi.exceptions.DatabaseError) 400 EXCEPT must be followed by ALL, DISTINCT, or "(" at [2:35]

[SQL: SELECT `set_example`.`id`
FROM `holyoke_test`.`set_example` EXCEPT SELECT `set_example`.`id`
FROM `holyoke_test`.`set_example`
WHERE `set_example`.`value` >= %(value_1:INT64)s]
[parameters: {'value_1': 5}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)

After this change we get:
[(2,), (3,), (6,)]

Similarly, with

expr = sa.select(table.c.id).intersect(
    sa.select(table.c.id).where(table.c.value >= 5),
)
conn.execute(expr).fetchall()

Before this change we get:

DatabaseError: (google.cloud.bigquery.dbapi.exceptions.DatabaseError) 400 Syntax error: Expected keyword ALL or keyword DISTINCT but got keyword SELECT at [2:45]

[SQL: SELECT `set_example`.`id`
FROM `holyoke_test`.`set_example` INTERSECT SELECT `set_example`.`id`
FROM `holyoke_test`.`set_example`
WHERE `set_example`.`value` >= %(value_1:INT64)s]
[parameters: {'value_1': 5}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)

After this change we get:
[(5,), (1,), (4,)]

It seems that this was discussed here with the maintainers of SA: sqlalchemy/sqlalchemy#11545, and the recommendation is for the dialect to implement the correct compound keywords for except_ and intersect

@aholyoke aholyoke requested review from a team as code owners July 13, 2024 21:01
@aholyoke aholyoke requested a review from hongalex July 13, 2024 21:01
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. labels Jul 13, 2024
@Linchin Linchin assigned Linchin and unassigned farhan0102 Jul 18, 2024
@Linchin Linchin added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 18, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 18, 2024
@Linchin Linchin added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 18, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 18, 2024
Copy link
Contributor

@Linchin Linchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing this error! LGTM.

@Linchin Linchin merged commit 80781ef into googleapis:main Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants