More robust processor from pretrained#42668
Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: audioflamingo3, auto, lasr, phi4_multimodal, pix2struct |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
ArthurZucker
left a comment
There was a problem hiding this comment.
Sorry LGTM later review, needs rebasing !
|
Thanks for the MRO walk, which fixes the inheritance case I reported. One small follow-up: @zucchini-nlp suggested in the issue thread allowing an explicit @classmethod
def get_attributes(cls):
if "attributes" in cls.__dict__ and cls.attributes is not None:
return list(cls.attributes)
# ... existing MRO-based detection ...Two concrete reasons: 1. The original case in this issue is still broken with the MRO walk alone. My actual code uses a custom (non-HF) tokenizer: class ActionTokenizerMixin:
def __init__(self, *args, action_tokenizer: Optional[BaseActionTokenizer] = None, **kwargs):
...
if not isinstance(argument, proper_class):
raise TypeError(
f"Received a {type(argument).__name__} for argument {argument_name}, "
f"but a {class_name} was expected."
)An explicit 2. Substring-based detection already needs core-level carve-outs.
It's the only such case today, but any future component whose name happens to overlap a modality substring (such as the non-HF custom tokenizer in this issue) will hit the same shape of problem. An explicit Happy to open a separate PR for this on top of yours. |
What does this PR do?
Fixes #42645
Depends on #42667
Add robustness to processor handling of subprocessors loading and saving by checking the signature of init in the parent classes as well.