cc @MekkCyber

Thanks for adding this @tanuj-rai!

I'm still not sure why we need this — could you include a script where this change is essential?

If it's just for supporting other quantization schemes that aren't implemented yet in Transformers, you can use the register_quantizer decorator — see this example: https://github.com/huggingface/transformers/blob/main/examples/quantization/custom_quantization_int8_example.py

Thank you @MekkCyber for the feedback! My goal of adding the hf_quantizer parameter to from_pretrained is to give users more flexibility, especially in cases where they already have a pre-instantiated HFQuantizer object. I think it can be really useful for more dynamic or reusable quantization setups that don’t neatly fit into the quantization_config pattern.

For example, if I want to create a quantizer that skips quantizing MoE gate layers or router components, I need to build that logic dynamically based on model structure or user input. This requires passing runtime config (like skip_keywords) to the quantizer instance — something not possible through the current quantization_config or registry-based system.

Here’s a minimal example that uses a subclass of HFQuantizer to selectively quantize layers:

# selective_quantizer.py
from transformers.utils.quantization_utils import HfQuantizer
from transformers import register_quantizer

class SelectiveQuantizer(HfQuantizer):
    def __init__(self, config, skip_keywords=None):
        super().__init__(config)
        self.skip_keywords = skip_keywords or []

    def quantize(self, model):
        for name, module in model.named_modules():
            if any(skip in name for skip in self.skip_keywords):
                continue
            if hasattr(module, "weight"):  # Simplified example
                module.weight.data = module.weight.data.half()  # Fake quantization
        model.quantized_by_selective = True
        return model

# use_quantizer.py
from transformers import AutoModelForCausalLM
from selective_quantizer import SelectiveQuantizer

custom_quantizer = SelectiveQuantizer(config={}, skip_keywords=["gate", "router"])

model = AutoModelForCausalLM.from_pretrained(
    "sshleifer/tiny-gpt2",
    hf_quantizer=custom_quantizer  # Only possible with proposed change
)

assert getattr(model, "quantized_by_selective", False), "Quantizer was not applied!"

Thanks for the reply @tanuj-rai and sorry for the delay. I see your point! I guess it's fine to add this if there is big community interest but we would rather not do that now, because it adds some maintenance complexity and forces us to keep the quantization api backward compatible, which we are not ready to do yet since we need to do some cleaning and refactoring

In the mean time i suggest you just use the decorator register_quantizer which should be as simple as passing hf_quantizer to from_pretrained

Thanks for the reply @tanuj-rai and sorry for the delay. I see your point! I guess it's fine to add this if there is big community interest but we would rather not do that now, because it adds some maintenance complexity and forces us to keep the quantization api backward compatible, which we are not ready to do yet since we need to do some cleaning and refactoring

In the mean time i suggest you just use the decorator register_quantizer which should be as simple as passing hf_quantizer to from_pretrained

No problem @MekkCyber. Thank you so much for the updates.