Description

We only gated the FP4 in the op level which is not enough.

we need to gate the entire dynamic_block_quantize.py

python examples/dynamo/torch_compile_resnet_example.py

  File "/home/ubuntu/Desktop/lan/jetson_py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/conversion/__init__.py", line 1, in <module>
    from . import (
  File "/home/ubuntu/Desktop/lan/jetson_py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/conversion/aten_ops_converters.py", line 13, in <module>
    from torch_tensorrt.dynamo.conversion import impl
  File "/home/ubuntu/Desktop/lan/jetson_py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/conversion/impl/__init__.py", line 1, in <module>
    from torch_tensorrt.dynamo.conversion.impl import (
  File "/home/ubuntu/Desktop/lan/jetson_py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/conversion/impl/dynamic_block_quantize.py", line 77, in <module>
    output_type: trt.DataType = trt.DataType.FP4,
AttributeError: type object 'tensorrt.tensorrt.DataType' has no attribute 'FP4'. Did you mean: 'FP8'?

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified