Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[CANN] Adapt to dynamically loadable backends mechanism #9970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 22, 2024
Merged

[CANN] Adapt to dynamically loadable backends mechanism #9970

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1657447
[CANN] Adapt to dynamically loadable backends mechanism
leo-pony Oct 21, 2024
4d4ae1c
Fix the Bug: inference running result is garbled in debug running mod…
leo-pony Oct 21, 2024
abf5be4
Handle the review comments of this pull request
leo-pony Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ggml/include/ggml-cann.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ extern "C" {
*/
#define GGML_CANN_MAX_DEVICES 16

GGML_API ggml_backend_reg_t ggml_backend_cann_reg(void);

/**
* @brief Initializes the CANN backend for a specified device.
*
Expand Down
9 changes: 8 additions & 1 deletion ggml/src/ggml-backend.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -561,6 +561,10 @@ void * ggml_backend_reg_get_proc_address(ggml_backend_reg_t reg, const char * na
# include "ggml-amx.h"
#endif

#ifdef GGML_USE_CANN
#include "ggml-cann.h"
#endif

struct ggml_backend_registry {
std::vector<ggml_backend_reg_t> backends;
std::vector<ggml_backend_dev_t> devices;
Expand All @@ -587,8 +591,11 @@ struct ggml_backend_registry {
#ifdef GGML_USE_AMX
register_backend(ggml_backend_amx_reg());
#endif
#ifdef GGML_USE_CANN
register_backend(ggml_backend_cann_reg());
#endif

// TODO: kompute, cann
// TODO: kompute

register_backend(ggml_backend_cpu_reg());
}
Expand Down
Loading
Loading