-
-
Notifications
You must be signed in to change notification settings - Fork 56.3k
Parsing quantized nodes does not rely on names #22531
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@zihaomu I propose to touch some of quantized models (per-layer test) to cover new case with test too. |
Hi @asmorkalov, thanks for your code reviewing. This patch wants to stop rely on the node name when we parse the quantized node. And it doesn't introduce new features. In my opinion, the existing test cases are enough. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for looking into this!
CV_Error(Error::StsNotImplemented, "Per-channel scales/zeropoints are not supported"); | ||
if (hasVariableInps) | ||
{ | ||
LayerInfo layerInfo = layer_id.find(node_proto.input(i))->second; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, you look for any real input layer and check if we created internal layer *Int8
? I mean, someone could rename the layer or add a new one that doesn't follow that convention. Maybe we're better off with layerParams.set("depth", CV_8S)
in each layer, or we could create a set of all integer layers. I think this should be discussed further with the rest of the team.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your code review. OpenCV DNN needs to allocate output blob according to depth type (for example the depth of CV_8S
will be allocated as int8 Mat.)
someone could rename the layer or add a new one that doesn't follow that convention.
It is possible. I agree with create a set of all integer layers
. I think a better way is for us to have both Int8
name endings and keep sets. The set only stores some special layers that do not follow the rules like Quantize
. For other layer type ending with Int8
, we can keep the current implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure relying on internal names is a good idea. Let's discuss this on Friday, maybe someone will come up with a better solution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @rogday, please take a look. I have refactored the code.
b3276e6
to
358edb7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
Ptr<Layer> preLayer = dstNet.getLayer(layerInfo.layerId); | ||
|
||
for (int i = 0; i < node_proto.output_size(); i++) | ||
if (layerInfo.depth == CV_8S && hasInt8Input(preLayer->type)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So if we insert unsupported at the moment layer(Gather, for example) after integer node, it will set the depth to CV_8S and corrupt the memory at runtime? (reading int8 as float in this case).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we have a set of ONNX layers instead? e.g. in this set we would put QuantizeLinear
, DequantizeLinear
, QLinearConv
, Transpose
, Reshape
, MaxPool
, etc.
And check that if we have integer inputs, current layer should be in this set, otherwise we throw an error, what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So if we insert unsupported at the moment layer(Gather, for example) after integer node, it will set the depth to CV_8S and corrupt the memory at runtime? (reading int8 as float in this case).
Yes, if we do this, it will corrupt. But this should not happen, Gather-like layer should output the same format as the input. So, if a Gather layer gets int8 input, it should be GatherInt8
. And if the type of GatherInt8 layer is Gather
, we can add the Gather
to the list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we have a set of ONNX layers instead? e.g. in this set we would put QuantizeLinear, DequantizeLinear, QLinearConv, Transpose, Reshape, MaxPool, etc.
And check that if we have integer inputs, the current layer should be in this set, otherwise we throw an error, what do you think?
I also considered a list of ONNX operators. Functionally, there is no difference between the two lists in essence, one is the original version of ONNX and the other is the mapped version of OpenCV.
Considering the future maintainability, indeed using the original version of ONNX operator names is better.
@zihaomu friendly reminder. |
@asmorkalov, Sorry for the late reply, I'm on vacation recently and will be back at work tomorrow. |
358edb7
to
d9eff7d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, one question.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! 👍
Discussed testing offline. Decided to merge without test. |
Related issues #22509 and #22442.
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.