HIP: disable rocwmma on gfx12 by default until rocm 7.0 #14202
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Nice, hopefully amd will add it in soon. Eventually as rocm updates this can probably be changed again. |
JohannesGaessler
approved these changes
Jun 16, 2025
|
With the version check I expect it to be fine since people will be running older versions for a while, 7.0 users would just get it automatically. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently rocwmma dose not support gfx12 in any released version of rocm, causeing build failures when llamacpp is built with GGML_HIP_ROCWMMA_FATTN enabled against gfx12.
This causes issues for one of our downstreams (koboldcpp) who want to build a fat binary with gfx8-gfx12 support and simultaneously use rocwmma where applicable. This detail has also caused various issues #14193 #13110 to be filed against llamacpp by confused users.
This pr thus disables rocwmma on gfx12 even when GGML_HIP_ROCWMMA_FATTN is set if rocm < 7.0 is used.
I have it on good authority that gfx12 rocwmma support will land in rocm 7.0 which is scheduled for release later this year.
As the rocwmma git repo has contained gfx12 support for some time, it has been possible to compile llamacpp against git rocwmma to get a working build for some time, to facilitate this, this pr also adds a GGML_HIP_FORCE_ROCWMMA_FATTN_GFX12 option to override the check for rocm 7.0 and try to use rocwmma regardless.