Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Disable tests for mixed opt_levels, add bitwise accurate test of parameters#520

Merged
mcarilli merged 4 commits into
NVIDIA:masterfrom
ptrblck:checkpointing
Oct 3, 2019
Merged

Disable tests for mixed opt_levels, add bitwise accurate test of parameters#520
mcarilli merged 4 commits into
NVIDIA:masterfrom
ptrblck:checkpointing

Conversation

@ptrblck
Copy link
Copy Markdown
Collaborator

@ptrblck ptrblck commented Oct 3, 2019

Increased atol to 1.5e-4 due to failing state_dict comparison for 'O0' and 'O2':

x = torch.tensor(0.8839)
torch.allclose(x.half().float(), x, atol=1e-4)
> False
torch.allclose(x.half().float(), x, atol=1.5e-4)
> True

@ptrblck ptrblck changed the title increase atol for Half-Float comparison to 1.5e-4 Disable tests for mixed opt_levels, add bitwise accurate test of parameters Oct 3, 2019
@ptrblck
Copy link
Copy Markdown
Collaborator Author

ptrblck commented Oct 3, 2019

First comment is deprecated.
The state_dict check now compares for bitwise accurate results and only for matching opt_level and restore_opt_level.

@mcarilli mcarilli merged commit 0b74bfd into NVIDIA:master Oct 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ptrblck @mcarilli