Add SetVariable and DeleteVariable to the Go SDK task client - #73084
Draft
henry3260 wants to merge 2 commits into
Draft
Add SetVariable and DeleteVariable to the Go SDK task client#73084henry3260 wants to merge 2 commits into
henry3260 wants to merge 2 commits into
Conversation
Go tasks could read Airflow Variables but not write or delete them, so the Go SDK did not meet the variable-read-write capability that language SDKs must support. The supervisor already handles PutVariable and DeleteVariable, so Go tasks only lacked a way to send them.
… read The note only mentioned AIRFLOW_VAR_* environment variables, but reads go through every secrets backend configured on the API server, so any of them can hide a value written from Go. The Java SDK docs already describe this precedence the same way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The language SDK specification (
contributing-docs/30_new_language_sdk.rst) requires every SDK to support thevariable-read-writecapability: a task must be able to read, write, and delete Airflow Variables. The Go SDK could only read them. The supervisor already handlesPutVariableandDeleteVariablefor Python tasks, and the matching message structs were already generated ingenmodels, but Go tasks had no API to send them. This is the Go counterpart of the Java change in #72676.What
go-sdk/sdk/sdk.go: addSetVariable(ctx, key, value, description string) errorandDeleteVariable(ctx, key string) errortoVariableClient, so they are also available throughsdk.Client.go-sdk/pkg/execution/client.go: implement both methods onCoordinatorClientby sendingPutVariable/DeleteVariableto the supervisor. An empty description is sent asnull, matching Python'sVariable.setdefault. Thedescriptionkey is always present on the wire, because the supervisor requires it.go-sdk/example/bundle/variablewrite/,go-sdk/example/bundle/main.go,go-sdk/dags/go_examples.py: a newvariable_write_dagexample whose Go task writes a Variable, then writes and deletes a scratch Variable.go-sdk/cmd/airflow-go-pack/pack_integration_test.go: add the new Dag to the expected bundle manifest.airflow-core/docs/authoring-and-scheduling/language-sdks/go.rst: document the new methods.Adding methods to
VariableClientmeans user-defined fakes that implement this interface must add the two methods. The example mocks in this repo are updated.Was generative AI tooling used to co-author this PR?