Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Discussion options

You must be logged in to vote

The current source says stale local files should be deleted.

S3DagBundle.refresh() calls S3Hook.sync_to_local_dir(..., delete_stale=True), and sync_to_local_dir() has a stale-file cleanup path that removes local files not present in the current S3 object list.

So I would first check the exact Airflow and apache-airflow-providers-amazon versions in the environment. If your installed provider version predates that delete_stale=True behavior, upgrading the Amazon provider may be the fix.

If you are already on a version with that code, then this is probably worth a small reproducible bug report: bucket prefix, bundle config, provider version, whether the deleted file is under the configured p…

Replies: 2 comments

Comment options

You must be logged in to vote
0 replies
Answer selected by ilgiznurgaliev
Comment options

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants