Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[pull] main from abetlen:main #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 41 additions & 42 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -143,46 +143,45 @@ if (LLAMA_BUILD)
)
endif()

# if (LLAVA_BUILD)
# if (LLAMA_CUBLAS OR LLAMA_CUDA)
# add_compile_definitions(GGML_USE_CUBLAS)
# add_compile_definitions(GGML_USE_CUDA)
# endif()
#
# if (LLAMA_METAL)
# add_compile_definitions(GGML_USE_METAL)
# endif()
#
# # Building llava
# add_subdirectory(vendor/llama.cpp/tools/mtmd)
# set_target_properties(llava_shared PROPERTIES OUTPUT_NAME "llava")
#
# if (WIN32)
# set_target_properties(llava_shared PROPERTIES CUDA_ARCHITECTURES OFF)
# endif()
# llama_cpp_python_install_target(llava_shared)
# if (WIN32)
# install(
# FILES $<TARGET_RUNTIME_DLLS:llava_shared>
# DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/llama_cpp/lib
# )
# install(
# FILES $<TARGET_RUNTIME_DLLS:llava_shared>
# DESTINATION ${SKBUILD_PLATLIB_DIR}/llama_cpp/lib
# )
# endif()
#
# # Fix for llava build: Add include directory for llama.h
# # Move these commands after the add_subdirectory call
# target_include_directories(llava PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
# target_include_directories(llava PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/ggml/include)
#
# if (BUILD_SHARED_LIBS)
# target_include_directories(llava_shared PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
# target_include_directories(llava_shared PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/ggml/include)
# endif()
#
# target_include_directories(llama-llava-cli PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
# target_include_directories(llama-minicpmv-cli PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
# endif()
if (LLAVA_BUILD)
if (LLAMA_CUBLAS OR LLAMA_CUDA)
add_compile_definitions(GGML_USE_CUBLAS)
add_compile_definitions(GGML_USE_CUDA)
endif()

if (LLAMA_METAL)
add_compile_definitions(GGML_USE_METAL)
endif()

# Building llava
add_subdirectory(vendor/llama.cpp/tools/mtmd)

if (WIN32)
set_target_properties(mtmd PROPERTIES CUDA_ARCHITECTURES OFF)
endif()
llama_cpp_python_install_target(mtmd)
if (WIN32)
install(
FILES $<TARGET_RUNTIME_DLLS:mtmd>
DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/llama_cpp/lib
)
install(
FILES $<TARGET_RUNTIME_DLLS:mtmd>
DESTINATION ${SKBUILD_PLATLIB_DIR}/llama_cpp/lib
)
endif()

# Fix for mtmd build: Add include directory for llama.h
# Move these commands after the add_subdirectory call
target_include_directories(mtmd PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
target_include_directories(mtmd PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/ggml/include)

if (BUILD_SHARED_LIBS)
target_include_directories(mtmd PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
target_include_directories(mtmd PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/ggml/include)
endif()

# target_include_directories(llama-llava-cli PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
# target_include_directories(llama-minicpmv-cli PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/vendor/llama.cpp/include)
endif()
endif()
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,7 @@ Below are the supported multi-modal models and their respective chat handlers (P
| [nanollava](https://huggingface.co/abetlen/nanollava-gguf) | `NanollavaChatHandler` | `nanollava` |
| [llama-3-vision-alpha](https://huggingface.co/abetlen/llama-3-vision-alpha-gguf) | `Llama3VisionAlphaChatHandler` | `llama-3-vision-alpha` |
| [minicpm-v-2.6](https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf) | `MiniCPMv26ChatHandler` | `minicpm-v-2.6` |
| [qwen2.5-vl](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-GGUF) | `Qwen25VLChatHandler` | `qwen2.5-vl` |

Then you'll need to use a custom chat handler to load the clip model and process the chat messages and images.

Expand Down
Loading