Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 #4956

jcwchen · 2023-03-02T17:38:20Z

Description

Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21

Motivation and Context

Supersede #4242

Signed-off-by: Chun-Wei Chen <[email protected]>

requirements.txt

Signed-off-by: Chun-Wei Chen <[email protected]>

…lds. Signed-off-by: Chun-Wei Chen <[email protected]>

Signed-off-by: Chun-Wei Chen <[email protected]>

…to jcw/protobuf4.21

justinchuby · 2023-04-05T03:08:27Z

setup.py

 TP_DIR = os.path.join(TOP_DIR, "third_party")
 CMAKE_BUILD_DIR = os.path.join(TOP_DIR, ".setuptools-cmake-build")
-PACKAGE_NAME = "onnx"
+PACKAGE_NAME = "onnx-protobufv21"


What does this do?

Good catch. This was for testing the built wheel with PyTorch... Here is the PR to fix this: #5089.

* [WIP] Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21 Signed-off-by: Chun-Wei Chen <[email protected]> * then Signed-off-by: Chun-Wei Chen <[email protected]> * Same as previous PR Signed-off-by: Chun-Wei Chen <[email protected]> * fix CIs Signed-off-by: Chun-Wei Chen <[email protected]> * 3.21.12 Signed-off-by: Chun-Wei Chen <[email protected]> * build protobuf from source on Linux Signed-off-by: Chun-Wei Chen <[email protected]> * static protobuf Signed-off-by: Chun-Wei Chen <[email protected]> * protoc --version Signed-off-by: Chun-Wei Chen <[email protected]> * $BUILD_TYPE Signed-off-by: Chun-Wei Chen <[email protected]> * printenv PATH Signed-off-by: Chun-Wei Chen <[email protected]> * path Signed-off-by: Chun-Wei Chen <[email protected]> * onnx-protobufv21 Signed-off-by: Chun-Wei Chen <[email protected]> * test 3.20.2 Signed-off-by: Chun-Wei Chen <[email protected]> * Protobuf 4 and Protobuf 3 will output different order of protobuf fields. Signed-off-by: Chun-Wei Chen <[email protected]> * skip Signed-off-by: Chun-Wei Chen <[email protected]> * %%script --no-raise-error false Signed-off-by: Chun-Wei Chen <[email protected]> * updated Signed-off-by: Chun-Wei Chen <[email protected]> * test lower protoc Signed-off-by: Chun-Wei Chen <[email protected]> * export CMAKE_ARGS="-DONNX_WERROR=ON -DONNX_USE_PROTOBUF_SHARED_LIBS=ON" Signed-off-by: Chun-Wei Chen <[email protected]> * fix Signed-off-by: Chun-Wei Chen <[email protected]> * still test protobuf==3.20.2 at least in a CI Signed-off-by: Chun-Wei Chen <[email protected]> * update Protobufs.ipynb Signed-off-by: Chun-Wei Chen <[email protected]> * ["nbreg"] = {"skip": True, "skip_reason": "Not ready for testing."} Signed-off-by: Chun-Wei Chen <[email protected]> * NBVAL_SKIP Signed-off-by: Chun-Wei Chen <[email protected]> * use " instead of ' Signed-off-by: Chun-Wei Chen <[email protected]> --------- Signed-off-by: Chun-Wei Chen <[email protected]> Signed-off-by: Sheil Kumar <[email protected]>

* Bump ossf/scorecard-action from 2.1.2 to 2.1.3 (#5084) Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.1.2 to 2.1.3. - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@e38b190...80e868c) --- updated-dependencies: - dependency-name: ossf/scorecard-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Sheil Kumar <[email protected]> * [CI] Fix weekly mac release CI failure and cover onnxruntime tests on AzurePipelines (#5057) * clean up CIs Signed-off-by: Chun-Wei Chen <[email protected]> * rename .yml instead of .yaml for consistency Signed-off-by: Chun-Wei Chen <[email protected]> * windows CI Signed-off-by: Chun-Wei Chen <[email protected]> --------- Signed-off-by: Chun-Wei Chen <[email protected]> Co-authored-by: Andreas Fehlner <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * Fix reference implementation for Pad and Reduce ops (#5008) * Fix reference implementation for Pad Signed-off-by: xadupre <[email protected]> * add unit test Signed-off-by: xadupre <[email protected]> * use parameterized Signed-off-by: xadupre <[email protected]> * check type Signed-off-by: xadupre <[email protected]> --------- Signed-off-by: xadupre <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * Add expected outputs for light models to make the backend test faster (#4960) * Add expected outputs for light models to make the backend test faster Signed-off-by: xadupre <[email protected]> * add expected outputs Signed-off-by: xadupre <[email protected]> * add missing outputs Signed-off-by: xadupre <[email protected]> * add missing expected output Signed-off-by: xadupre <[email protected]> * lint Signed-off-by: xadupre <[email protected]> * lint Signed-off-by: xadupre <[email protected]> * skip failing test for onnxruntime Signed-off-by: xadupre <[email protected]> * lint Signed-off-by: xadupre <[email protected]> * use range as well Signed-off-by: xadupre <[email protected]> * fix a bug Signed-off-by: xadupre <[email protected]> * exception Signed-off-by: xadupre <[email protected]> * fix light models by lower the fill value of ConstantOfShape nodes Signed-off-by: xadupre <[email protected]> * lint Signed-off-by: xadupre <[email protected]> * fix type issue in deform implementation Signed-off-by: xadupre <[email protected]> * exclude deformconv from onnxruntime backend test series Signed-off-by: xadupre <[email protected]> * disable test when numpy is not recent enough Signed-off-by: xadupre <[email protected]> * move one test Signed-off-by: xadupre <[email protected]> * avoid python 3.7 Signed-off-by: xadupre <[email protected]> * add missing import Signed-off-by: xadupre <[email protected]> * fix import Signed-off-by: xadupre <[email protected]> * add missing license Signed-off-by: xadupre <[email protected]> * add copyright for binary file Signed-off-by: xadupre <[email protected]> --------- Signed-off-by: xadupre <[email protected]> Co-authored-by: Chun-Wei Chen <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * merge batch dimension fix to STFT Signed-off-by: Sheil Kumar <[email protected]> * Add clang tidy review in Lint CI (#5041) * add clang-tidy review Signed-off-by: Chun-Wei Chen <[email protected]> * more configs Signed-off-by: Chun-Wei Chen <[email protected]> * add post yml Signed-off-by: Chun-Wei Chen <[email protected]> * rename Signed-off-by: Chun-Wei Chen <[email protected]> * post Signed-off-by: Chun-Wei Chen <[email protected]> * orkflow_dispatch Signed-off-by: Chun-Wei Chen <[email protected]> * Post clang-tidy review comments Signed-off-by: Chun-Wei Chen <[email protected]> * remove workflow Signed-off-by: Chun-Wei Chen <[email protected]> * test Signed-off-by: Chun-Wei Chen <[email protected]> * revert demo Signed-off-by: Chun-Wei Chen <[email protected]> --------- Signed-off-by: Chun-Wei Chen <[email protected]> Co-authored-by: Andreas Fehlner <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 (#4956) * [WIP] Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21 Signed-off-by: Chun-Wei Chen <[email protected]> * then Signed-off-by: Chun-Wei Chen <[email protected]> * Same as previous PR Signed-off-by: Chun-Wei Chen <[email protected]> * fix CIs Signed-off-by: Chun-Wei Chen <[email protected]> * 3.21.12 Signed-off-by: Chun-Wei Chen <[email protected]> * build protobuf from source on Linux Signed-off-by: Chun-Wei Chen <[email protected]> * static protobuf Signed-off-by: Chun-Wei Chen <[email protected]> * protoc --version Signed-off-by: Chun-Wei Chen <[email protected]> * $BUILD_TYPE Signed-off-by: Chun-Wei Chen <[email protected]> * printenv PATH Signed-off-by: Chun-Wei Chen <[email protected]> * path Signed-off-by: Chun-Wei Chen <[email protected]> * onnx-protobufv21 Signed-off-by: Chun-Wei Chen <[email protected]> * test 3.20.2 Signed-off-by: Chun-Wei Chen <[email protected]> * Protobuf 4 and Protobuf 3 will output different order of protobuf fields. Signed-off-by: Chun-Wei Chen <[email protected]> * skip Signed-off-by: Chun-Wei Chen <[email protected]> * %%script --no-raise-error false Signed-off-by: Chun-Wei Chen <[email protected]> * updated Signed-off-by: Chun-Wei Chen <[email protected]> * test lower protoc Signed-off-by: Chun-Wei Chen <[email protected]> * export CMAKE_ARGS="-DONNX_WERROR=ON -DONNX_USE_PROTOBUF_SHARED_LIBS=ON" Signed-off-by: Chun-Wei Chen <[email protected]> * fix Signed-off-by: Chun-Wei Chen <[email protected]> * still test protobuf==3.20.2 at least in a CI Signed-off-by: Chun-Wei Chen <[email protected]> * update Protobufs.ipynb Signed-off-by: Chun-Wei Chen <[email protected]> * ["nbreg"] = {"skip": True, "skip_reason": "Not ready for testing."} Signed-off-by: Chun-Wei Chen <[email protected]> * NBVAL_SKIP Signed-off-by: Chun-Wei Chen <[email protected]> * use " instead of ' Signed-off-by: Chun-Wei Chen <[email protected]> --------- Signed-off-by: Chun-Wei Chen <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * add reuse badge (#5085) * Update README.md --------- Signed-off-by: Andreas Fehlner <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * Update reuse.yml (#5087) Signed-off-by: Andreas Fehlner <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> * onnx (#5089) Signed-off-by: Chun-Wei Chen <[email protected]> Co-authored-by: Andreas Fehlner <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Sheil Kumar <[email protected]> Signed-off-by: Chun-Wei Chen <[email protected]> Signed-off-by: xadupre <[email protected]> Signed-off-by: Andreas Fehlner <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chun-Wei Chen <[email protected]> Co-authored-by: Andreas Fehlner <[email protected]> Co-authored-by: Xavier Dupré <[email protected]>

jcwchen added 2 commits March 2, 2023 09:35

[WIP] Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21

69a3c83

Signed-off-by: Chun-Wei Chen <[email protected]>

Merge remote-tracking branch 'upstream/main' into jcw/protobuf4.21

45926a6

jcwchen added topic: build Issues related to ONNX builds and packages module: CI pipelines Issues related to the CI pipeline run release CIs Use this label to trigger release tests in CI dependencies Pull requests that update a dependency file labels Mar 2, 2023

jcwchen requested a review from a team as a code owner March 2, 2023 17:38

xadupre reviewed Mar 2, 2023

View reviewed changes

requirements.txt Outdated Show resolved Hide resolved

jcwchen added 16 commits March 2, 2023 09:41

then

8c40bfc

Signed-off-by: Chun-Wei Chen <[email protected]>

Same as previous PR

f55786b

Signed-off-by: Chun-Wei Chen <[email protected]>

fix CIs

d65d055

Signed-off-by: Chun-Wei Chen <[email protected]>

3.21.12

f4e5d55

Signed-off-by: Chun-Wei Chen <[email protected]>

build protobuf from source on Linux

11d32da

Signed-off-by: Chun-Wei Chen <[email protected]>

static protobuf

aadcec9

Signed-off-by: Chun-Wei Chen <[email protected]>

protoc --version

2eb2a39

Signed-off-by: Chun-Wei Chen <[email protected]>

$BUILD_TYPE

3b8bffc

Signed-off-by: Chun-Wei Chen <[email protected]>

printenv PATH

cb1d3fa

Signed-off-by: Chun-Wei Chen <[email protected]>

path

d7e49be

Signed-off-by: Chun-Wei Chen <[email protected]>

onnx-protobufv21

f1d03c4

Signed-off-by: Chun-Wei Chen <[email protected]>

test 3.20.2

8debe39

Signed-off-by: Chun-Wei Chen <[email protected]>

Protobuf 4 and Protobuf 3 will output different order of protobuf fie…

177615c

…lds. Signed-off-by: Chun-Wei Chen <[email protected]>

skip

bc0b1db

Signed-off-by: Chun-Wei Chen <[email protected]>

%%script --no-raise-error false

3395540

Signed-off-by: Chun-Wei Chen <[email protected]>

updated

8c51e6a

Signed-off-by: Chun-Wei Chen <[email protected]>

This was referenced Mar 6, 2023

Update ONNX submodule from ONNX 1.13.1 with Protobuf 4.21 updates pytorch/pytorch#96138

Closed

[WIP] Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21 #4242

Closed

Support for latest protobuf versions > 4 #4971

Closed

jcwchen added 3 commits March 23, 2023 08:28

test lower protoc

1cca6ee

Signed-off-by: Chun-Wei Chen <[email protected]>

export CMAKE_ARGS="-DONNX_WERROR=ON -DONNX_USE_PROTOBUF_SHARED_LIBS=ON"

862333a

Signed-off-by: Chun-Wei Chen <[email protected]>

fix

ee8600c

Signed-off-by: Chun-Wei Chen <[email protected]>

still test protobuf==3.20.2 at least in a CI

4907b60

Signed-off-by: Chun-Wei Chen <[email protected]>

jcwchen force-pushed the jcw/protobuf4.21 branch from 9e612f9 to 4907b60 Compare March 23, 2023 21:18

Merge branch 'main' into jcw/protobuf4.21

cd2f594

jcwchen added this to the 1.14.0 milestone Mar 28, 2023

Merge branch 'main' into jcw/protobuf4.21

08f8f08

jcwchen changed the title ~~[WIP] Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21~~ Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21 Mar 29, 2023

jcwchen changed the title ~~Upgrade Protobuf to v21.1 to accept Python Protobuf 4.21~~ Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 Mar 29, 2023

update Protobufs.ipynb

648b23f

Signed-off-by: Chun-Wei Chen <[email protected]>

gramalingam approved these changes Apr 3, 2023

View reviewed changes

jcwchen added 8 commits April 3, 2023 15:08

Merge branch 'main' into jcw/protobuf4.21

dcbfbdc

Signed-off-by: Chun-Wei Chen <[email protected]>

Merge branch 'main' into jcw/protobuf4.21

1508790

["nbreg"] = {"skip": True, "skip_reason": "Not ready for testing."}

c5441c0

Signed-off-by: Chun-Wei Chen <[email protected]>

NBVAL_SKIP

53856bf

Signed-off-by: Chun-Wei Chen <[email protected]>

Merge branch 'main' into jcw/protobuf4.21

af14ee5

use " instead of '

a0a6c0f

Signed-off-by: Chun-Wei Chen <[email protected]>

Merge branch 'jcw/protobuf4.21' of https://github.com/jcwchen/onnx in…

edef005

…to jcw/protobuf4.21

Merge branch 'main' into jcw/protobuf4.21

88e2048

jcwchen enabled auto-merge (squash) April 4, 2023 22:45

jcwchen merged commit a220317 into onnx:main Apr 4, 2023

justinchuby reviewed Apr 5, 2023

View reviewed changes

jcwchen mentioned this pull request Apr 5, 2023

Use correct package name onnx #5089

Merged

jcwchen deleted the jcw/protobuf4.21 branch April 5, 2023 03:12

jcwchen mentioned this pull request May 15, 2023

Upgrade ONNX to 1.14.0 conda-forge/onnx-feedstock#94

Merged

5 tasks

adamjstewart mentioned this pull request Jan 6, 2024

(py-)onnx: new version 1.14.{0,1}, 1.15.0 spack/spack#41877

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 #4956

Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 #4956

Uh oh!

jcwchen commented Mar 2, 2023 •

edited

Loading

Uh oh!

Uh oh!

justinchuby Apr 5, 2023

Uh oh!

jcwchen Apr 5, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 #4956

Upgrade Protobuf to v21.12 to accept Python Protobuf 4.21 #4956

Uh oh!

Conversation

jcwchen commented Mar 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

Uh oh!

justinchuby Apr 5, 2023

Choose a reason for hiding this comment

Uh oh!

jcwchen Apr 5, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jcwchen commented Mar 2, 2023 •

edited

Loading