Fill DagBag from dag_folder setting for upgrade rules#14588
Fill DagBag from dag_folder setting for upgrade rules#14588kaxil merged 9 commits intoapache:v1-10-stablefrom
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
|
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
|
@kaxil @ephraimbuddy @XD-DENG can you help with the tests? The failures are on rules that I did not change. Also I don't understand why it passes for postgres 13 + py 3.7 but not for postgres 13 + py 2.7 |
|
Hi @kenfinster , most likely the error has nothing to do with your changes, because the same errors already exist in the latest commit in branch But on the other hand, the most recent change you made in 4b7f4a7 is not making sense to me. I guess you were making a random try? Otherwise may you please clarify it? Thanks. |
|
@XD-DENG There was a failure with: So i change the assertion. Thinking of it again maybe better way is to remove this test all together. It tries to mock a dag bag but this PR makes it redundant. |
@kenfinster Can you remove your last commit 4b7f4a7. This is a known issue(see here #14493) which I think should be fixed in a separate PR. |
|
@ephraimbuddy done. |
|
@kenfinster I have rebased your PR on latest (after merging #14678 and #14680) v1-10-stable -- hopefully the CI failures should be fixed. |
|
The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason. |
|
Awesome work, congrats on your first merged pull request! |
|
@kenfinster How were you setting the dag folder? I was unable to reproduce this behaviour with Airflow 1.10.14 and upgrade-check 1.2.0 |
For some reason:
BaseOperatorMetaclassRule
NoAdditionalArgsInOperatorsRule
Don't consider the dag folder set by the user. This causes issues when the user needs to set the dag folder with environment variable.
This fix makes the rules to take the path from the settings.
@kaxil following the chat on slack.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.