Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add transport parameter to CloudRunHook and CloudRunExecuteJobOperator#60394

Merged
shahar1 merged 6 commits into
apache:mainfrom
arjav1528:dev-issue-60389
Jan 26, 2026
Merged

Add transport parameter to CloudRunHook and CloudRunExecuteJobOperator#60394
shahar1 merged 6 commits into
apache:mainfrom
arjav1528:dev-issue-60389

Conversation

@arjav1528
Copy link
Copy Markdown
Contributor

Fixes #60389

Summary

Adds support for specifying the transport protocol (REST or gRPC) in CloudRunHook, CloudRunAsyncHook, CloudRunExecuteJobOperator, and CloudRunJobFinishedTrigger to resolve 404 errors in Docker environments where gRPC transport fails.

Problem

CloudRunExecuteJobOperator fails with 404 errors (NotFound: 404 Requested entity was not found) when running in Docker containers, even though the same Cloud Run jobs work correctly with gcloud CLI. This occurs because the Google Cloud Python client defaults to gRPC transport, which can fail in Docker environments due to network configurations, proxy settings, or firewall restrictions.

Solution

Added an optional transport parameter to:

  • CloudRunHook - allows specifying 'rest' or 'grpc' transport for JobsClient
  • CloudRunAsyncHook - allows specifying transport for JobsAsyncClient
  • CloudRunExecuteJobOperator - passes transport parameter to the hook
  • CloudRunJobFinishedTrigger - passes transport parameter to async hook for deferrable operations

…for API requests. This allows users to specify 'rest' or 'grpc' transport methods across CloudRunHook, CloudRunAsyncHook, CloudRunExecuteJobOperator, and CloudRunJobFinishedTrigger classes
@boring-cyborg boring-cyborg Bot added area:providers provider:google Google (including GCP) related issues labels Jan 11, 2026
Comment thread providers/google/src/airflow/providers/google/cloud/triggers/cloud_run.py Outdated
…rt method in CloudRunHook, CloudRunAsyncHook, CloudRunExecuteJobOperator, and CloudRunJobFinishedTrigger classes. Adjusted related tests to reflect this change
…, defaulting to 'grpc' for backward compatibility. Adjusted related tests to accommodate this change
Comment thread providers/google/tests/unit/google/cloud/hooks/test_cloud_run.py Outdated
…CloudRunExecuteJobOperator to allow None as a valid option, enabling automatic transport selection. Updated related tests to validate this change
Copy link
Copy Markdown
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM with a small comment - please fix it and I'll approve.

CC: @VladaZakharova @MaksYermak @olegkachur-e

Comment thread providers/google/src/airflow/providers/google/cloud/triggers/cloud_run.py Outdated
…to 'grpc' when None is provided, ensuring backward compatibility. Updated related logic for consistency
@arjav1528 arjav1528 requested a review from uranusjr January 24, 2026 22:54
@arjav1528
Copy link
Copy Markdown
Contributor Author

@uranusjr could you please approve the changes and merge the PR

@shahar1 shahar1 merged commit 68f05b3 into apache:main Jan 26, 2026
89 checks passed
@arjav1528 arjav1528 deleted the dev-issue-60389 branch January 26, 2026 22:20
shreyas-dev pushed a commit to shreyas-dev/airflow that referenced this pull request Jan 29, 2026
shashbha14 pushed a commit to shashbha14/airflow that referenced this pull request Feb 2, 2026
jason810496 pushed a commit to abhijeets25012-tech/airflow that referenced this pull request Feb 3, 2026
@MaksYermak
Copy link
Copy Markdown
Contributor

Hello @shahar1!
This PR broke the deferrable mode for CloudRunExecuteJobOperator could you revert this PR?
Here is the screenshot from the last run of the system test:
Screenshot From 2026-02-04 15-25-13

@shahar1
Copy link
Copy Markdown
Contributor

shahar1 commented Feb 4, 2026

Hello @shahar1!
This PR broke the deferrable mode for CloudRunExecuteJobOperator could you revert this PR?
Here is the screenshot from the last run of the system test:
Screenshot From 2026-02-04 15-25-13

Thanks for informing, I'll take care of it later on - I had merged it before our discussion regarding the system tests.
As I'm also the the release manager of the next provider wave, I'll ensure that it's get reverted by then.
I'll be happy to discuss with your team regarding the integration of the system tests to the repo's CI, so we could run specific system twsts before merging / getting alerted on Airflow's slack channel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CloudRunExecuteJobOperator fails with 404 in Docker (gRPC transport)

5 participants