Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

Enable bulking test on windows#14392

Merged
szha merged 6 commits into
apache:masterfrom
DickJC123:debug_bulking_test_on_windows
Mar 12, 2019
Merged

Enable bulking test on windows#14392
szha merged 6 commits into
apache:masterfrom
DickJC123:debug_bulking_test_on_windows

Conversation

@DickJC123
Copy link
Copy Markdown
Contributor

@DickJC123 DickJC123 commented Mar 11, 2019 •
edited
Loading

Description

This PR is a response to #14368 and is now ready for merging. The first step here was to recreate the Windows CI failure, along with additional debugging output to show whether the intended bulking environment variable settings were making it to the point of use. I learned that the test's repeated setting and altering of environment variables was not getting through to the C++ runtime under Windows, which apparently gets a copy of the environment at process start-up. The fix was to spawn a new test process for each setting of different environment variables, as is already done for the gluon version of these same tests.

Thanks to @perdasilva for identifying this issue.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • [X ] Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • [X ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@karan6181
Copy link
Copy Markdown
Contributor

karan6181 commented Mar 12, 2019 •
edited
Loading

@mxnet-label-bot add [CI, Test, Windows, pr-work-in-progress]]

@DickJC123
Copy link
Copy Markdown
Contributor Author

DickJC123 commented Mar 12, 2019 •
edited
Loading

If this latest commit passes CI, it will be ready for merging. The problem is that when one performs in Python:

os.environ[<key>] = <value>

this effects what the C++ backend sees only on Linux. Under Windows, some additional (and possibly windows-version-specific) work would be required to push the environment change to the msvcrt, which otherwise gets a copy of the environment at process start-up. My preferred fix here is to resort to the more bomb-proof approach of setting environment variables, then spawning a new process to perform the test (although admittedly this is bypassed on Python2 due to lack of spawn support).

@DickJC123 DickJC123 changed the title Debug bulking test on windows [WIP] Enable bulking test on windows Mar 12, 2019
@DickJC123
Copy link
Copy Markdown
Contributor Author

DickJC123 commented Mar 12, 2019

Ready for your review. @perdasilva @eric-haibin-lin @szha

@szha szha merged commit 66c74cc into apache:master Mar 12, 2019
vdantu pushed a commit to vdantu/incubator-mxnet that referenced this pull request Mar 31, 2019
@DickJC123 @vdantu
Enable bulking test on windows (apache#14392)
e974ac2 
* Reenable test_operator_gpu.py:test_bulking.

* Add temporary debug output.

* Add test_gluon_gpu.py:test_bulking to test spawn approach.

* Reorder testing to see test_gluon_gpu.py:test_bulking result.

* Change test_operator_gpu.py:test_bulking to use spawn approach.

* Remove diagnostic output and revert windows ci changes.
nswamy pushed a commit that referenced this pull request Apr 5, 2019
@DickJC123 @nswamy
Enable bulking test on windows (#14392)
438aa6e 
* Reenable test_operator_gpu.py:test_bulking.

* Add temporary debug output.

* Add test_gluon_gpu.py:test_bulking to test spawn approach.

* Reorder testing to see test_gluon_gpu.py:test_bulking result.

* Change test_operator_gpu.py:test_bulking to use spawn approach.

* Remove diagnostic output and revert windows ci changes.
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
@DickJC123
Enable bulking test on windows (apache#14392)
f8f625f 
* Reenable test_operator_gpu.py:test_bulking.

* Add temporary debug output.

* Add test_gluon_gpu.py:test_bulking to test spawn approach.

* Reorder testing to see test_gluon_gpu.py:test_bulking result.

* Change test_operator_gpu.py:test_bulking to use spawn approach.

* Remove diagnostic output and revert windows ci changes.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

CI pr-work-in-progress PR is still work in progress Test Windows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@DickJC123 @karan6181 @szha @marcoabreu