-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Set download retries and log download exceptions for installer failures #7600
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set download retries and log download exceptions for installer failures #7600
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for tackling one of the causes of the pipeline instability! 🚀
I only have a small nitpick, nothing blocking a merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly LGTM 👍 I agree with what @alexrashed said, + one small thing that I wanted to clarify (but is not blocking in any way)
if os.path.getsize(tmp_archive) <= 0: | ||
raise Exception("Failed to download archive from %s: . Retries exhausted", archive_url) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question: can this actually ever be a negative value? The docs state that in case of inaccessibility, OSError is returned.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TBH, just copied this from L185.
Like you said, probably not, but I guess we still wouldn't want it if for some reason this would happen on some esoteric system 🤷♂️
Motivation
PR was motivated by failing downloads of ffmpeg where the Installer tries to extract the empty installer file instead of just re-trying to download the archive first. It also didn't log any meaningful exceptions in this case since it wasn't a
requests.exceptions.ReadTimeout
Changes