When I tried to quantize belows,
python -m qai_hub_models.models.qwen3_0_6b.quantize \
--checkpoint Qwen/Qwen3-0.6B \
--output-dir ./quantized_model
The operation sequence is as follows. Finally, for the foundation model PTQ running on SM8850
i understand that adaptation model creation -> preparation model creation -> quantization should be performed However, I am inquiring because it seems to be omitted in the process of confirmation.
I would appreciate it if you could explain about it.
quantize() call
Based on the parsed factor, the quantize() function responsible for the actual quantization logic is executed.
FP model load
Load the original (floating point) model into memory (CPU) with FP_Model.from_pretrained().
Apply model adaptation (model_adaptations)
SHAQwen3Attention: After converting to Conv‐in‐place, Split‐Head Attention is implemented.
QCQwen3MLP / QCQwen3ForCausalLM: Increase memory / operation efficiency by replacing with Linear with ConvInplaceLinear.
This step is defined in src/qai_hub_models/models/_shared/qwen3/model_adaptations.py.
Prepare calibration data
collect input data by the number of samples specified as model_quant.get_calibration_data().
convert dataset_entries_to_dataloader() to PyTorch DataLoader and use it for quantization afterwards.
...
When I tried to quantize belows,
The operation sequence is as follows. Finally, for the foundation model PTQ running on SM8850
i understand that adaptation model creation -> preparation model creation -> quantization should be performed However, I am inquiring because it seems to be omitted in the process of confirmation.
I would appreciate it if you could explain about it.