Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix(bigquery): preserve catalog in partition metadata lookup#40200

Merged
rusackas merged 3 commits into
apache:masterfrom
puneetdixit200:fix/bigquery-partition-cross-project
Jun 24, 2026
Merged

fix(bigquery): preserve catalog in partition metadata lookup#40200
rusackas merged 3 commits into
apache:masterfrom
puneetdixit200:fix/bigquery-partition-cross-project

Conversation

@puneetdixit200

@puneetdixit200 puneetdixit200 commented May 18, 2026

Copy link
Copy Markdown
Contributor

SUMMARY

Preserve the BigQuery project/catalog when looking up time partition metadata.

get_time_partition_column previously called client.get_table() with only dataset.table, so the BigQuery client
could fall back to the credentials/default project and return a 404 for cross-project datasets. The lookup now uses
project.dataset.table when Table.catalog is available.

Fixes #40131

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Not applicable.

TESTING INSTRUCTIONS

Automated checks:

  • venv\Scripts\python.exe -m pytest tests\unit_tests\db_engine_specs\test_bigquery.py -q
  • venv\Scripts\python.exe -m ruff format --check superset\db_engine_specs\bigquery.py tests\unit_tests\db_engine_specs\test_bigquery.py
  • venv\Scripts\python.exe -m ruff check superset\db_engine_specs\bigquery.py tests\unit_tests\db_engine_specs\test_bigquery.py
  • git diff --check

Manual verification:

  1. Configure a BigQuery database where the credentials project differs from the dataset project.
  2. Select a schema/dataset from the dataset project.
  3. Open metadata for a time-partitioned table.
  4. Confirm metadata loads without a 404 from the credentials/default project.

ADDITIONAL INFORMATION

Copilot AI review requested due to automatic review settings May 18, 2026 01:03
@dosubot dosubot Bot added the data:connect:googlebigquery Related to BigQuery label May 18, 2026
@bito-code-review

bito-code-review Bot commented May 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Code Review Agent Run #49e3b4

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: 4c5a6ac..4c5a6ac
    • superset/db_engine_specs/bigquery.py
    • tests/unit_tests/db_engine_specs/test_bigquery.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at [email protected].

Documentation & Help

AI Code Review powered by Bito Logo

Copilot AI reviewed May 18, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Fixes BigQuery partition column lookup to include the project (catalog) in the fully-qualified table reference, ensuring partition metadata is retrieved from the correct project rather than the connection's default.

Changes:

  • Prepend table.catalog to the BigQuery table reference passed to client.get_table when available.
  • Add a unit test verifying the catalog is included in the table reference.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
superset/db_engine_specs/bigquery.py Build table reference as catalog.schema.table when catalog is present.
tests/unit_tests/db_engine_specs/test_bigquery.py New test asserting catalog-qualified table reference is used.

@codecov

codecov Bot commented May 18, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.36%. Comparing base (584d417) to head (43a1129).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
superset/db_engine_specs/bigquery.py 0.00% 4 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #40200      +/-   ##
==========================================
- Coverage   64.40%   64.36%   -0.05%     
==========================================
  Files        2653     2653              
  Lines      145325   145128     -197     
  Branches    33548    33467      -81     
==========================================
- Hits        93593    93407     -186     
+ Misses      50034    50023      -11     
  Partials     1698     1698
Flag Coverage Δ
hive 39.23% <0.00%> (-0.01%) ⬇️
mysql 57.97% <0.00%> (-0.01%) ⬇️
postgres 58.04% <0.00%> (-0.01%) ⬇️
presto 40.81% <0.00%> (-0.01%) ⬇️
python 59.47% <0.00%> (-0.01%) ⬇️
sqlite 57.69% <0.00%> (-0.01%) ⬇️
unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

rusackas
rusackas approved these changes Jun 23, 2026
View reviewed changes

@rusackas rusackas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a clean fix. Just updated the branch to see if CI will pass.

Without the catalog prefix, client.get_table("dataset.table") resolves against the client's default project, so partition metadata gets silently missed for tables in another BigQuery project, and the qualified project.dataset.table reference is exactly right. The guard keeps the no-catalog path unchanged and the unit test pins the resolved reference.

Will merge soon if all goes well!

@github-actions

github-actions Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

⚠️ Translation Regression Detected

A source change in this PR renamed or reworded strings, invalidating existing translations (they are now #, fuzzy) in fr. Please resolve the affected .po files before merging.

Note: intentionally deleting a translatable string is not a regression and is not flagged here — only translations invalidated by a renamed/reworded source string are.

Language Fuzzy before Fuzzy after New
fr 1004 1014 +10

How to fix

1. Install dependencies (if not already set up):

pip install -r superset/translations/requirements.txt
sudo apt-get install gettext   # or: brew install gettext

2. Re-extract strings and sync .po files:

./scripts/translations/babel_update.sh

This rewrites superset/translations/messages.pot from the current source files and merges the changes into every .po file. Strings whose msgid changed will be marked #, fuzzy.

3. Resolve the fuzzy entries in the affected language files (fr):

grep -n '#, fuzzy' superset/translations/<lang>/LC_MESSAGES/messages.po

For each fuzzy entry, either rewrite the msgstr to match the new string and remove the #, fuzzy line, or clear the msgstr to "" if you cannot provide a translation.

4. Commit your changes to the .po files.

@rusackas rusackas merged commit 7b9bcdd into apache:master Jun 24, 2026
60 checks passed
shantanukhond pushed a commit to shantanukhond/superset that referenced this pull request Jun 24, 2026
@puneetdixit200 @rusackas @shantanukhond
fix(bigquery): preserve catalog in partition metadata lookup (apache#…
a6f978c 
…40200)

Co-authored-by: Puneet Dixit <[email protected]>
Co-authored-by: Evan Rusackas <[email protected]>
@bito-code-review

bito-code-review Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Bito Automatic Review Skipped – PR Already Merged

Bito scheduled an automatic review for this pull request, but the review was skipped because this PR was merged before the review could be run.
No action is needed if you didn't intend to review it. To get a review, you can type /review in a comment and save it

devin-ai-integration Bot pushed a commit to cognition-demo/superset that referenced this pull request Jun 29, 2026
@puneetdixit200 @rusackas @devin-ai-integration
fix(bigquery): preserve catalog in partition metadata lookup (apache#…
62e774b 
…40200)

Co-authored-by: Puneet Dixit <[email protected]>
Co-authored-by: Evan Rusackas <[email protected]>
@dosubot dosubot Bot mentioned this pull request Jul 9, 2026
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@rusackas rusackas rusackas approved these changes

Assignees

No one assigned

Labels

data:connect:googlebigquery Related to BigQuery size/S

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BigQuery: get_time_partition_column uses 2-part table reference, causing 404 for cross-project datasets

3 participants

@puneetdixit200 @rusackas