Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[ONNX] Refactoring serialization of ONNX initializers to be name-based (Resubmission)#17830

Closed
spandantiwari wants to merge 14 commits into
pytorch:masterfrom
spandantiwari:spandantiwari/params_with_names
Closed

[ONNX] Refactoring serialization of ONNX initializers to be name-based (Resubmission)#17830
spandantiwari wants to merge 14 commits into
pytorch:masterfrom
spandantiwari:spandantiwari/params_with_names

Conversation

@spandantiwari
Copy link
Copy Markdown

@houseroad - this is the resubmission of #17420, as suggested.

@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Mar 9, 2019
@houseroad
Copy link
Copy Markdown
Member

Apparently, this break several internal pipelines... I need more time to investigate what's going on.

Copy link
Copy Markdown
Member

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put it on hold before the internal tests pass.

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@spandantiwari
Copy link
Copy Markdown
Author

spandantiwari commented Mar 10, 2019

@houseroad - OK. Could it be because of the order the initializers are written to the ONNX file, because that may change for some graphs with this change? But that should be a one-time update to the expect file.
Let me know if I can help.

@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from 6b45432 to f48e2ec Compare March 16, 2019 00:27
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Copy Markdown
Member

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is only triggered when we set input_names and using non-protobuf serialization format (e.g., ZIP_ARCHIVE). We should add a test case to guard such situation.

Comment thread torch/onnx/utils.py Outdated
@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from f48e2ec to fb28b81 Compare March 27, 2019 00:09
@spandantiwari
Copy link
Copy Markdown
Author

Fixed the issue with setting input_names and using non-protobuf serialization format (e.g., ZIP_ARCHIVE). Also added a test for guarding against that.

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from 15ac29e to 451894e Compare March 27, 2019 18:52
Comment thread torch/onnx/utils.py Outdated
@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from 451894e to 1a8220b Compare March 28, 2019 00:58
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Copy Markdown
Member

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a test case to cover the input_names conflict with existing parameter names.

Something like model has 2 parameters and 1 input.
export(model, input_names=['input1', 'p2']) and the second parameter's name is p2, and it will be renamed to p2.1 after exporting to onnx.

Comment thread torch/onnx/utils.py Outdated
Comment thread torch/onnx/utils.py Outdated
Comment thread torch/onnx/utils.py Outdated
@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from 1a8220b to d5f87f5 Compare March 28, 2019 21:43
@spandantiwari
Copy link
Copy Markdown
Author

Moved the params_dict creation code to the end as suggested. Added a new test point that checks tests the deliberate parameter name clobber.

@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from d5f87f5 to bf4ae98 Compare March 28, 2019 22:28
Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@spandantiwari
Copy link
Copy Markdown
Author

@houseroad - the failures seem unrelated. I have seen some of them in previous runs as well.

Copy link
Copy Markdown
Member

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Finally, we fixed all the problems.

Left some nits.

Comment thread test/onnx/test_pytorch_onnx_caffe2.py Outdated
Comment thread test/onnx/test_pytorch_onnx_caffe2.py Outdated
Comment thread test/onnx/test_pytorch_onnx_caffe2.py Outdated
Comment thread test/onnx/test_pytorch_onnx_caffe2.py Outdated
Comment thread torch/csrc/jit/python_ir.cpp Outdated
@spandantiwari spandantiwari force-pushed the spandantiwari/params_with_names branch from bf4ae98 to 13c5516 Compare March 29, 2019 18:00
@spandantiwari
Copy link
Copy Markdown
Author

@houseroad - Great to hear that all tests are passing. Thanks for your tremendous help!
I have updated the PR based on the feedback.

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@houseroad merged this pull request in 1240327.

laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
…bmission) (pytorch#17830)

Summary:
houseroad - this is the resubmission of pytorch#17420, as suggested.
Pull Request resolved: pytorch#17830

Reviewed By: zrphercule

Differential Revision: D14398714

Pulled By: houseroad

fbshipit-source-id: bda475f1ae8a5273ebdb0f6883fc66036c29d326
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

oncall: jit Add this issue/PR to JIT oncall triage queue open source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants