Fix HF tests by switching them to upstream testing models#6261
Fix HF tests by switching them to upstream testing models#6261sw005320 merged 3 commits intoespnet:masterfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request is a great step towards improving test maintenance by switching to upstream Hugging Face testing models. The changes are mostly straightforward replacements of model names. I found one potential issue in the test setup where the vocabulary size for the generated token list might be too small for one of the new test models (BloomForCausalLM), which could lead to test failures or incorrect behavior. I've added a specific comment with a suggestion to address this.
for more information, see https://pre-commit.ci
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6261 +/- ##
===========================================
+ Coverage 23.88% 56.77% +32.88%
===========================================
Files 888 889 +1
Lines 84196 84361 +165
===========================================
+ Hits 20114 47898 +27784
+ Misses 64082 36463 -27619
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thanks!! |
I created copies of the upstream HF testing models in the past, and some of them became incompatible with the recent versions of HF Transformers because of the parameter naming or configuration format. The migration process for old models is not quite clear to me, because HF created new versions of models from scratch (I found only one migrated example: https://huggingface.co/hf-internal-testing/tiny-random-gpt2/commits/main).
The reason to create my copies of the HF testing models was to support PyTorch<1.6, because all original models used the new PyTorch
torch.saveformat. Now this is not required, so all tests can be switched to the upstream testing models, which are maintained by HF.