Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: support sync batchnorm#151

Merged
NOBLES5E merged 33 commits into
masterfrom
sync_bn
Aug 26, 2021
Merged

feat: support sync batchnorm#151
NOBLES5E merged 33 commits into
masterfrom
sync_bn

Conversation

@Tengxu-Sun
Copy link
Copy Markdown
Contributor

Not official.

@Tengxu-Sun Tengxu-Sun marked this pull request as draft August 5, 2021 03:04
@Tengxu-Sun Tengxu-Sun linked an issue Aug 5, 2021 that may be closed by this pull request
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Comment thread bagua/torch_api/contrib/sync_batchnorm.py
Comment thread bagua/torch_api/contrib/sync_batchnorm.py Outdated
Copy link
Copy Markdown
Contributor

@NOBLES5E NOBLES5E left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

Comment thread tests/contrib/test_sync_bn.py
Comment thread tests/contrib/test_sync_bn.py
Comment thread tests/contrib/test_sync_bn.py
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py
Comment thread tests/contrib/test_sync_bn.py Outdated
Comment thread tests/contrib/test_sync_bn.py Outdated
@Tengxu-Sun Tengxu-Sun marked this pull request as ready for review August 26, 2021 08:09
@Tengxu-Sun Tengxu-Sun requested a review from NOBLES5E August 26, 2021 08:15
Comment thread tests/contrib/test_sync_bn.py Outdated
"LOCAL_WORLD_SIZE": str(nprocs),
"MASTER_ADDR": "127.0.0.1",
"MASTER_PORT": str(find_free_port()),
"BAGUA_SERVICE_PORT": str(find_free_port()),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated find_free_port func in another pr. please check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wangraying Which one to use?

Comment thread bagua/torch_api/contrib/sync_batchnorm.py
Comment thread tests/internal/common_utils.py Outdated
return port
except OSError:
port += 1
raise IOError("no free ports") No newline at end of file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blackfmt] reported by reviewdog 🐶

Suggested change
raise IOError("no free ports")
raise IOError("no free ports")

@NOBLES5E NOBLES5E merged commit 3937288 into master Aug 26, 2021
@NOBLES5E NOBLES5E deleted the sync_bn branch August 26, 2021 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement SyncBN

3 participants