Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Dear llama.cpp team,

I'm writing to submit a feature request: please consider adding official support for the AMD Ryzen AI platform NPU .

Currently, AMD has provided an initial implementation in their open-source project:
🔗 https://github.com/amd/RyzenAI-SW/tree/1.2.0/example/transformers/ext/llama.cpp

However, based on community feedback, this implementation may still have room for improvement in terms of performance and usability. Integrating this functionality into the official llama.cpp repository could help improve cross-platform compatibility.

It's also worth noting that different platforms support varying quantization precisions:

STX (e.g., HX370) supports both W8A16 and W4ABF16
PHX (e.g., 7840HS), HPT (e.g., 8845HS) support W4ABF16, but NOT support W8A16
For more details, please refer to:
🔗 https://github.com/amd/RyzenAI-SW/blob/1.2.0/example/transformers/models/llm/docs/README.md

Thank you for your hard work and contributions! We look forward to better support for Ryzen AI NPUs in the future.

Motivation

Integrating this functionality into the official llama.cpp repository could help improve cross-platform compatibility.

Possible Implementation

No response