fix(processing): Filter kwargs in ProcessorMixin call to prevent Type…#41606
fix(processing): Filter kwargs in ProcessorMixin call to prevent Type…#41606CodersAcademy006 wants to merge 1 commit into
Conversation
|
cc @zucchini-nlp! I'm not sure about this because some of the submodules may just have |
| input_data, input_kwargs = attribute_to_kwargs[attribute_name] | ||
| if input_data is not None and attribute is not None: | ||
| attribute_output = attribute(input_data, **kwargs[input_kwargs]) |
There was a problem hiding this comment.
in this line kwargs are filtered already and we are obtaining kwargs[input_kwargs]. That should work for all models if if there's a TypeError anywhere, I'd suggest to see what has gone wrong with specific model
There was a problem hiding this comment.
The model where this issue is occurring is CSM-1B, so it might be related to how that particular architecture handles its input kwargs.
…cit signatures `_merge_kwargs` routes kwargs into per-modality buckets based on TypedDict membership. Several kwargs (e.g. `pad_to_multiple_of`) are declared in more than one modality TypedDict (TextKwargs and AudioKwargs), so they land in every matching bucket. When a concrete sub-processor has an explicit `__call__` signature without a **kwargs catch-all it then receives unexpected keyword arguments and raises a TypeError. Fix: before calling each sub-processor, inspect its `__call__` signature. - If a VAR_KEYWORD parameter exists the processor already handles unknown keys itself, so all modality kwargs are forwarded unchanged. - Otherwise, filter modality kwargs to the set of params the sub-processor explicitly accepts. Adds two regression tests in ProcessorMixinKwargsFilterTest that reproduce the failure with a strict-signature feature extractor and confirm the **kwargs path continues to work unchanged. Fixes huggingface#41598
|
Hi, Severity: remediation recommended | Category: reliability How to fix: Add signature() fallback path Agent prompt to fix - you can give this to your LLM of choice:
Found by Qodo code review |
This PR fixes an issue where multimodal processors could raise a
TypeErrorwhen called with arguments specific to one modality. TheProcessorMixin.__call__method was passing all keyword arguments to each sub-processor (tokenizer, feature extractor, etc.), causing a crash if a processor received an argument it didn't recognize (e.g.,EncodecFeatureExtractorreceivingpad_to_multiple_offrom the tokenizer).This fix resolves the problem by filtering the keyword arguments before they are passed to each sub-processor. It uses Python's
inspectmodule to get the valid parameters for each processor's__call__method and only passes the arguments that match. This preventsTypeErrors and makes the processor logic more robust.Fixes #41598