Harmonize HF environment variables + deprecate use_auth_token - #6066
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Why did you think that removing this arg is a good idea? |
patrickvonplaten
left a comment
There was a problem hiding this comment.
That looks great - thanks for all the effort!
|
@patrickvonplaten Why do you think that removing use_auth_token arg is a good idea? |
|
@patrickvonplaten also why do you approve this commit with not all checks passed? |
|
@kopyl Argument Regarding CI checks I guess the failing error was unrelated to this PR. |
|
@patrickvonplaten thank you very much. I either did not read properly the description, or there was no description originally. |
…gface#6066) * Harmonize HF environment variables + deprecate use_auth_token * fix import * fix
…gface#6066) * Harmonize HF environment variables + deprecate use_auth_token * fix import * fix
…gface#6066) * Harmonize HF environment variables + deprecate use_auth_token * fix import * fix
(Similar to huggingface/transformers#27564. Sorry in advance for the size of the diff, I hope it'll be ok :) )
This PR aims at harmonizing environment variables in the HF ecosystem (following up on huggingface/huggingface_hub#1786). I also took the time to review the integration of
huggingface_hubintodiffusersto update to latest "standard". I tried to break nothing -staying at iso-feature- but the review will be much appreciated 😄. The overall goal is simply to have less duplication in the codebase.List of changes:
use_auth_tokenin favor oftokeneverywhere and in a backward-compatible way. To do do I used the@validate_hf_hub_argsdecorator that checks which ofuse_auth_tokenand/ortokenare passed. For now usinguse_auth_tokendo not raise a warning but it will in the future. The goal is to slowly adapt scripts before deprecating officially. And the deprecation will only have to happen inhuggingface_hubthanks to the decorator.use_auth_token=True. This is not necessary anymore since the token is always passed to the server by default (already the case since ~1y ago).local_files_onlytoHF_HUB_OFFLINEby default. This is already done inhuggingface_hub(less duplicated code)cache_dirtoDIFFUSERS_CACHEby default. This is already done inhuggingface_hub(less duplicated code)DIFFUSERS_CACHEwhich is strictly equal toHF_HUB_CACHE(shared with other libraries)HF_HUB_OFFLINEandDISABLE_TELEMETRYfrom environment variables but usehuggingface_hubvalues insteadhuggingface_hub.constants.HF_HOMEinstead ofhuggingface_hub.constants.hf_cache_homehuggingface_hub.get_full_repo_name(instead of defining it in diffusers)_get_model_file(that one was for myself - it looks like a heavily used method)And that's it. Mostly cleaning some stuff in a backward-compatible way to remove duplicated work. Sorry in advance for the tedious review 🙈