Introduce lowbit quantized linear MPS kernels #954
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/954
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 6661971 with merge base 0f6bae5 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
manuelcandales
added a commit
to manuelcandales/ao
that referenced
this pull request
Sep 26, 2024
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ └── test/
│ │ └── (directly test mps kernel implementations)
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Differential Revision: D63342895
9ad395e to
e06802b
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
manuelcandales
added a commit
to manuelcandales/ao
that referenced
this pull request
Sep 26, 2024
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ └── test/
│ │ └── (directly test mps kernel implementations)
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Differential Revision: D63342895
e06802b to
e783565
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
kimishpatel
requested changes
Oct 2, 2024
Contributor
kimishpatel
left a comment
There was a problem hiding this comment.
Left quite a few comments. Lets address those. At high level
- Figure out plan to how we gonna read .metal file. Likely similar to what torch does but see what MLX does too
- Better test cases
kimishpatel
requested changes
Oct 2, 2024
Contributor
kimishpatel
left a comment
There was a problem hiding this comment.
Left quite a few comments. Lets address those. At high level
- Figure out plan to how we gonna read .metal file. Likely similar to what torch does but see what MLX does too
- Better test cases
e783565 to
b4943fe
Compare
manuelcandales
added a commit
to manuelcandales/ao
that referenced
this pull request
Oct 9, 2024
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ └── test/
│ │ └── (directly test mps kernel implementations)
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Differential Revision: D63342895
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
manuelcandales
added a commit
to manuelcandales/ao
that referenced
this pull request
Oct 10, 2024
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ └── test/
│ │ └── (directly test mps kernel implementations)
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Differential Revision: D63342895
b4943fe to
6c16c82
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
malfet
approved these changes
Oct 10, 2024
| const std::string& source) { | ||
| NSError* error = nil; | ||
| MTLCompileOptions* options = [MTLCompileOptions new]; | ||
| [options setLanguageVersion:MTLLanguageVersion3_1]; |
Contributor
There was a problem hiding this comment.
You probably want to disable fast math here as well...
| ext_modules=[ | ||
| CppExtension( | ||
| name="torchao_mps_ops", | ||
| sources=["register.mm"], |
Contributor
There was a problem hiding this comment.
I think you either need to package your kernels into the wheel, or using bin2c just convert them to headers. Or convert them into a metallib
manuelcandales
added a commit
to manuelcandales/ao
that referenced
this pull request
Oct 10, 2024
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── codegen/
│ │ └── gen_metal_shader_lib.py
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ ├── test/
│ │ └── (directly test mps kernel implementations)
│ └── metal.yaml
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Reviewed By: malfet
Differential Revision: D63342895
6c16c82 to
730561d
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
Summary:
The following is the directory structure of the submitted code under torchao
```
experimental/
├── kernels/
│ └── mps/
│ ├── codegen/
│ │ └── gen_metal_shader_lib.py
│ ├── metal/
│ │ └── (metal shaders)
│ ├── src/
│ │ └── (tensor agnostic mps kernel implementations)
│ ├── test/
│ │ └── (directly test mps kernel implementations)
│ └── metal.yaml
└── ops/
└── mps/
├── register.mm
├── setup.py
└── test/
└── (test torch custom ops)
```
Reviewed By: malfet
Differential Revision: D63342895
730561d to
6661971
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D63342895 |
jainapurva
pushed a commit
that referenced
this pull request
Oct 15, 2024
Differential Revision: D63342895 Pull Request resolved: #954
yanbing-j
pushed a commit
to yanbing-j/ao
that referenced
this pull request
Dec 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
The following is the directory structure of the submitted code under torchao
Differential Revision: D63342895