Backward function for torch.cdist#17173
Conversation
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
| _(aten, _pad_packed_sequence) \ | ||
| _(aten, _pdist_backward) \ | ||
| _(aten, _pdist_forward) \ | ||
| _(aten, _cdist_backward) \ |
There was a problem hiding this comment.
don't think you actually need this.
| f_args_tensor = deepcopy(unpack_variables(f_args_variable)) | ||
| run_functional_checks(self, "test_cdist", "cdist", f, | ||
| True, f_args_variable, f_args_tensor) | ||
| self.assertTrue(gradcheck(f, f_args_variable, eps=1e-6, atol=PRECISION)) |
There was a problem hiding this comment.
you don't need this, it's already checked in run_functional_checks, it's just there in the above test as explained in the comments.
| REGISTER_DISPATCH(cdist_backward_stub, &cdist_backward_kernel_impl); | ||
|
|
||
| }} // at::native | ||
| }} // at::native No newline at end of file |
There was a problem hiding this comment.
nit: I think there should be a newline here.
| False, f_args_variable, f_args_tensor) | ||
| self.assertTrue(gradcheck(lambda a, b: torch.cat((a, b)), f_args_variable, eps=1e-6, atol=PRECISION)) | ||
|
|
||
| def test_cdist(self): |
There was a problem hiding this comment.
can you move this above or below all the cat tests?
| f = lambda a, b: torch.cdist(a, b, p) | ||
| f_args_tensor = deepcopy(unpack_variables(f_args_variable)) | ||
| run_functional_checks(self, "test_cdist", "cdist", f, | ||
| True, f_args_variable, f_args_tensor) |
There was a problem hiding this comment.
hmm...this exposed a pretty serious bug in our testing. This test is running double backwards tests and passing, but it shouldn't be because you didn't write a double backwards function.
I believe what is happening is this:
- because you don't have an entry in derivatives.yaml for your backwards function (i.e. a double backwards definition), it is assumed to be composed of differentiable functions
- it's not actually composed of differentiable functions
- the testing doesn't catch this because it only looks at differentiable outputs. But your function doesn't create any differentiable outputs.
This should be testable, though. Basically, this should only be valid if the numerical gradient of the function is 0. Can you add checks for that? (You should probably do it in a separate commit, there may be other functions that exhibit the same problem).
There was a problem hiding this comment.
Added test to EXCLUDE_GRADGRADCHECK_BY_TEST_NAME list
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
…of files. Removed debuging info
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
gchanan
left a comment
There was a problem hiding this comment.
can you fix the underlying issue with gradgradcheck? Which is basically that if none of the outputs require_grad, we don't actually check them. We should instead check that their numerical gradients are 0.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ifedan is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Summary: Pull Request resolved: pytorch/pytorch#17173 Differential Revision: D14111482 Pulled By: ifedan fbshipit-source-id: d72cfd53c29d0f8cf5f8ad1148d14f3d5abd938e
|
This diff fails lint |
Summary: Pull Request resolved: pytorch#17173 Differential Revision: D14111482 Pulled By: ifedan fbshipit-source-id: d72cfd53c29d0f8cf5f8ad1148d14f3d5abd938e
No description provided.