Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Convolution transpose #1648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Dec 3, 2021
Merged

Convolution transpose #1648

merged 5 commits into from
Dec 3, 2021

Conversation

@hunse
Copy link
Contributor

@hunse hunse commented Oct 19, 2020 •
edited
Loading

Motivation and context:

Transpose convolution is a fairly general way to do different types of upsampling in deep networks. Adding this transform allows many more types of deep networks to be implemented in Nengo, including many types of object detection networks.

How has this been tested?

I added a test that ensures that the ConvTransposeInc op is the transpose of ConvInc. Since ConvInc is tested against the Scipy implementation, if all tests pass then ConvTransposeInc (and the underlying code we've borrowed from np-conv2d) should be working fine. I've also added tests for the API (mostly piggy-backing off the existing Convolution test).

How long should this take to review?

  • Average (neither quick nor lengthy)

Where should a reviewer start?

Start with the ConvolutionTranspose transform, since this is the API (it's fairly similar to the Convolution transform). Then, the new test in builder/tests/test_transforms.py is the main test that's added to test the underlying implementation. The code in _vendors/np-conv2d is copied from np-conv2d, with the modifications listed in the conv2d_gradx docstring.

Types of changes:

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have read the CONTRIBUTING.rst document.
  • I have updated the documentation accordingly.
  • I have included a changelog entry.
  • I have added tests to cover my changes.
  • I have run the test suite locally and all tests passed.

Still to do:

  • Changelog entries
  • Check that it's showing in the API docs properly

This was referenced Oct 19, 2020
Merged
Merged
@hunse hunse force-pushed the convolution-transpose branch 2 times, most recently from e263a36 to 03dda9e Compare November 18, 2020 20:48
drasmuss
drasmuss reviewed Nov 30, 2020
View reviewed changes
Copy link
Member

@drasmuss drasmuss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a couple minor comments/questions.

@drasmuss
Copy link
Member

drasmuss commented Dec 1, 2020

Fixups added. I was thinking I'd squash all the convolution-related commits (so everything except the node init ordering and test_seeding), unless you think there are any that are important to keep separate?

@hunse
Copy link
Contributor Author

hunse commented Dec 1, 2020

The fixups look good. Squashing all the convolution ones is fine by me.

@hunse hunse force-pushed the convolution-transpose branch from 73b9ab1 to 3689657 Compare December 7, 2020 22:41
@hunse
Copy link
Contributor Author

hunse commented Dec 7, 2020

@drasmuss Got this rebased onto master, and made the appropriate f-string changes (I didn't see any OrderedDicts or places for pathlib). I don't think all the fixups necessarily point to the right base commit, but if you're squashing together all the convolution stuff anyway it should be fine.

@hunse
Copy link
Contributor Author

hunse commented Dec 9, 2020

I've fixed the alias problem in nengo/nengo-sphinx-theme#70. This now requires that.

@hunse hunse mentioned this pull request Jan 11, 2021
Merged
1 task
@xchoo xchoo force-pushed the convolution-transpose branch 2 times, most recently from 5388828 to f9fa742 Compare May 26, 2021 03:05
@hunse hunse force-pushed the convolution-transpose branch 2 times, most recently from 05676b4 to 788e607 Compare November 25, 2021 14:40
@hunse
Copy link
Contributor Author

hunse commented Nov 25, 2021

This is ready for another look @drasmuss, when you get a chance.

hunse added 3 commits November 25, 2021 11:52
@hunse
Fix pylint errors in test_threading.py
7869227
@hunse
Do not print fails in test_seeding
8c5df35
@hunse
Fix param init order for Node
aa86dc5 
Otherwise, if `output` is initialized before `size_out`, a function
`output` will be run unnecessarily to determine `size_out`.
@hunse hunse force-pushed the convolution-transpose branch from db5c26e to b2cd1af Compare November 25, 2021 16:53
tbekolay
tbekolay requested changes Dec 2, 2021
View reviewed changes
Copy link
Member

@tbekolay tbekolay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, but this PR adds a lot of noise to the test suite that we can hopefully pare down before merging. Also, we've brought in TensorFlow as an optional test dependency, which isn't ideal, because we don't want to really want to mark it as an optional dependency because it doesn't do anything except for run tests, but we also don't want to mark it as a test dependency because it's a very big dependency that tests a small fraction of the codebase. Is there a simple-ish way to implement those tests without TensorFlow? If not, I think we will probably want to figure out some way to ensure that these tests run at least as part of the release process, as otherwise they won't get run regularly.

@hunse
Copy link
Contributor Author

hunse commented Dec 2, 2021

As for including TensorFlow, I'm happy to have that only run on release. I don't see a simple way to get around it; the whole point of vendorizing that conv2d library was that it's a pure Numpy implementation of convolution, so we don't need TensorFlow, but I then discovered that it wasn't always giving the same results as TensorFlow so I wanted to test against it. Now that I've fixed those bugs, it's probably not necessary for the tests to run every build, but it's still nice to have them if we're changing the code in the future (or if TensorFlow changes).

@tbekolay
Copy link
Member

tbekolay commented Dec 2, 2021

Huh ... I'm not sure why exactly but this branch is causing the Appveyor build to run out of memory. Only for Python 3.6 though so ... not sure if it's worth debugging much.

hunse
hunse commented Dec 3, 2021
View reviewed changes
hunse added 2 commits December 3, 2021 09:56
@hunse @tbekolay
Create ChannelShape from space and channels
0215576
@hunse @tbekolay
Add ConvolutionTranspose transform
7a08d8e 
We also add the Conv and ConvTranspose aliases as they will be
familiar to TensorFlow users.

This commit also cleans up some of the original Convolution
implementation, raises an exception when Convolution has empty
output, and tests Convolution against TensorFlow.
@tbekolay tbekolay force-pushed the convolution-transpose branch from 41a52a0 to 7a08d8e Compare December 3, 2021 15:58
tbekolay
tbekolay approved these changes Dec 3, 2021
View reviewed changes
Copy link
Member

@tbekolay tbekolay left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the tests, made a story for running with TF on release, and cleaned up the history. I'll merge on successful CI.

@tbekolay tbekolay merged commit 7a08d8e into master Dec 3, 2021
@tbekolay tbekolay deleted the convolution-transpose branch December 3, 2021 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@drasmuss drasmuss drasmuss left review comments

@tbekolay tbekolay tbekolay approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hunse @drasmuss @tbekolay