Codestin Search App

Dev-iL · 2026-02-28T11:16:30Z

Summary

Add rule AIR304 that flags runtime-varying function calls (e.g., datetime.now(), pendulum.now(), random.randint(), uuid.uuid4()) used as arguments in Airflow DAG/task constructors. These calls cause the serialized DAG hash to change on every parse, creating infinite DAG versions in the dag_version and serialized_dag tables, leading to unbounded database growth and eventual OOM conditions.

Reference: apache/airflow#59430

The rule checks:

DAG(...) constructors and @dag(...) decorator calls
Operator and sensor constructors (via is_airflow_builtin_or_provider)
@task(...) decorator calls

It recursively inspects keyword argument values through binary/unary ops, dicts, lists, sets, tuples, and f-strings to find calls to known runtime-varying functions from datetime, pendulum, time, uuid, and random.

Test Plan

cargo test -p ruff_linter -- airflow::tests — all 43 tests pass, including the new AIR304 test case with 17 violation and 6 non-violation scenarios covering direct calls, binary ops, default_args dicts, f-strings, operators, sensors, decorators, and non-airflow calls.

Related: apache/airflow#43176
CC: @Lee-W @sjyangkevin @wjddn279

@dag

Using runtime-varying values (like `datetime.now()`) as arguments to Airflow DAG or task constructors causes the serialized DAG hash to change on every parse, creating infinite DAG versions in the database. This rule detects such calls in DAG constructors, @dag decorators, operator/sensor constructors, and @task decorators, recursively checking through binary ops, dicts, lists, sets, tuples, and f-strings. Co-Authored-By: Claude Opus 4.6 <[email protected]>

astral-sh-bot · 2026-02-28T11:25:41Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+48 -0 violations, +0 -0 fixes in 1 projects; 55 projects unchanged)

apache/airflow (+48 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview --select ALL

+ airflow-core/tests/unit/models/test_dag.py:3116:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3161:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3172:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3197:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3215:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3230:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3258:64: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3326:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3335:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3347:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3370:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3394:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3420:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3439:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3450:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3481:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3510:63: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/models/test_dag.py:3524:60: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/serialization/test_dag_serialization.py:1859:56: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/serialization/test_serialized_objects.py:254:52: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ airflow-core/tests/unit/serialization/test_serialized_objects.py:366:55: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/celery/tests/integration/celery/test_celery_executor.py:254:85: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/celery/tests/integration/celery/test_celery_executor.py:306:85: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_job.py:493:74: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_job.py:64:59: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py:113:59: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py:1679:55: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py:1748:74: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_pod.py:313:24: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/databricks/tests/unit/databricks/operators/test_databricks.py:1220:53: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/databricks/tests/unit/databricks/operators/test_databricks.py:2478:67: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/databricks/tests/unit/databricks/operators/test_databricks.py:446:53: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/databricks/tests/unit/databricks/operators/test_databricks.py:785:53: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/google/tests/system/google/cloud/composer/example_cloud_composer.py:244:26: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/google/tests/system/google/cloud/composer/example_cloud_composer.py:257:26: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/openlineage/tests/unit/openlineage/utils/test_selective_enable.py:35:86: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/openlineage/tests/unit/openlineage/utils/test_selective_enable.py:45:80: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/yandex/tests/system/yandex/example_yandexcloud_dataproc.py:97:73: AIR304 `uuid.uuid4()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/yandex/tests/unit/yandex/operators/test_dataproc.py:79:31: AIR304 `datetime.today()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/yandex/tests/unit/yandex/operators/test_yq.py:49:31: AIR304 `datetime.today()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ providers/ydb/tests/unit/ydb/operators/test_ydb.py:105:31: AIR304 `datetime.today()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/bases/test_operator.py:320:64: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/bases/test_operator.py:345:66: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/bases/test_operator.py:402:73: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/bases/test_operator.py:451:66: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/bases/test_operator.py:476:66: AIR304 `datetime.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/definitions/test_taskgroup.py:653:79: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation
+ task-sdk/tests/task_sdk/definitions/test_taskgroup.py:825:46: AIR304 `pendulum.now()` produces a value that changes at runtime; using it in a DAG or task argument causes infinite DAG version creation

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
AIR304	48	48	0	0	0

astral-sh-bot bot assigned ntBre Feb 28, 2026

ntBre added rule Implementing or modifying a lint rule preview Related to preview mode features labels Mar 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[`airflow`] Flag runtime-varying values in DAG/task constructor arguments (AIR304)#23631

[`airflow`] Flag runtime-varying values in DAG/task constructor arguments (AIR304)#23631
Dev-iL wants to merge 1 commit intoastral-sh:mainfrom
Dev-iL:2602/airflow/runtime

Dev-iL commented Feb 28, 2026

Uh oh!

astral-sh-bot bot commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Dev-iL commented Feb 28, 2026

Summary

Test Plan

Uh oh!

astral-sh-bot bot commented Feb 28, 2026

ruff-ecosystem results

Linter (stable)

Linter (preview)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`ruff-ecosystem` results