[PyTorch] Add lr scheduler #1305
Conversation
|
Job d2l-en/PR-1305/2 is complete. |
|
Job d2l-en/PR-1305/1 is complete. |
|
Job d2l-en/PR-1305/5 is complete. |
|
Job d2l-en/PR-1305/6 is complete. |
|
Job d2l-en/PR-1305/7 is complete. |
|
Job d2l-en/PR-1305/8 is complete. |
|
|
||
| ```{.python .input} | ||
| scheduler = lr_scheduler.CosineScheduler(20, warmup_steps=5, base_lr=0.5, | ||
| scheduler = lr_scheduler.CosineScheduler(20, warmup_steps=5, base_lr=0.3, |
There was a problem hiding this comment.
PyTorch implementation was not converging regularly somehow when base_lr was 0.5. So I changed it across frameworks for consistency.
There was a problem hiding this comment.
That's ok. Just give me a heads up if you change original mx code :)
There was a problem hiding this comment.
Sure! I mentioned it here. Will make it more explicit in the future if mxnet code is changed.
| #@tab tensorflow | ||
| scheduler = SquareRootScheduler(1.0) | ||
| #@tab all | ||
| scheduler = SquareRootScheduler(lr=0.1) |
There was a problem hiding this comment.
Here's another example of original code modification. Please make sure that similar results are obtained.
|
Should we merge this now? |
Description of changes:
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.