Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Define TaskInstanceKey in task-sdk to support client server separation#60776

Merged
amoghrajesh merged 8 commits into
apache:mainfrom
astronomer:ti-key-to-sdk
Jan 24, 2026
Merged

Define TaskInstanceKey in task-sdk to support client server separation#60776
amoghrajesh merged 8 commits into
apache:mainfrom
astronomer:ti-key-to-sdk

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh commented Jan 19, 2026


Was generative AI tooling used to co-author this PR?
  • No

Defining TaskInstanceKey in task-sdk as exposing it as a part of the SDK public API. This enables providers to use TaskInstanceKey without depending on airflow-core models (older path).

For context, TaskInstanceKey provides task information (dag_id, task_id, run_id, map_index, try_number) which is used now for:

  • Public REST API
  • Operator link interfaces used by 100+ providers

Why not follow this comment: https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/executors/workloads.py#L77 and actually do it? There are a few blockers to that.

  • While TaskInstance model now has a UUID primary key, TaskInstanceKey still remains as the interface for public APIs and provider contracts due to backcompat (airflow 2.x doesn't have uuid)
  • We should be able to do this only when min version of airflow for providers reaches 3.x
  • XCOM API could be a potential blocker here too because dag_id, run_id, task_id is extensively used there

Whats next?

  • Add this shim layer to compat/sdk.py and have providers consume from that

@amoghrajesh amoghrajesh requested review from ashb, eladkal, kaxil, potiuk and uranusjr and removed request for ashb and kaxil January 19, 2026 13:07
@amoghrajesh amoghrajesh self-assigned this Jan 19, 2026
@amoghrajesh amoghrajesh added this to the Airflow 3.2.0 milestone Jan 19, 2026
Comment thread task-sdk/src/airflow/sdk/types.py
Comment thread task-sdk/src/airflow/sdk/bases/operatorlink.py
@amoghrajesh
Copy link
Copy Markdown
Contributor Author

This one needs a bit more work to fix CI

@amoghrajesh
Copy link
Copy Markdown
Contributor Author

Umm this databricks failure will need some ground work, its using the old signature to support AF2

@amoghrajesh amoghrajesh requested a review from kaxil January 22, 2026 06:06
Copy link
Copy Markdown
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High lgtm but do have some questions

@amoghrajesh amoghrajesh merged commit d32d58c into apache:main Jan 24, 2026
130 checks passed
@amoghrajesh amoghrajesh deleted the ti-key-to-sdk branch January 24, 2026 14:34
suii2210 pushed a commit to suii2210/airflow that referenced this pull request Jan 26, 2026
shreyas-dev pushed a commit to shreyas-dev/airflow that referenced this pull request Jan 29, 2026
jhgoebbert pushed a commit to jhgoebbert/airflow_Owen-CH-Leung that referenced this pull request Feb 8, 2026
@amoghrajesh
Copy link
Copy Markdown
Contributor Author

cc: @atul-astronomer no targetted testing needed but compat is enough

choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants