Tags: LongLeCE/llama.cpp
Tags
Add Q2_0 quantization: type definition and CPU backend (ggml-org#24448)
common: Set optimal default thread count for ppc ( linux as well as A… …IX) (ggml-org#25237)
Fix stale tensor-split params for draft models (ggml-org#24814) * meta: fix tensor split metadata for GQA attention * Tidied the code a bit to match existing style * Revert "Tidied the code a bit to match existing style" This reverts commit b90c6c6. * Reverted the ggml-backend-meta asset hack.
llama : add guard for K/V rotation input when buffer is unallocated (g… …gml-org#25215) llm_graph_input_attn_kv::set_input and llm_graph_input_attn_kv_iswa::set_input call set_input_k_rot / set_input_v_rot whenever the rotation tensor pointer is non-null, but the tensor's buffer can be unallocated (NULL) when a graph only stores K/V without attending -- e.g. DFlash speculative decoding's KV-injection pass. set_input_k_rot then calls ggml_backend_buffer_is_host() on a NULL buffer and aborts with GGML_ASSERT(buffer). Guard the four k_rot/v_rot inputs with the same "&& ->buffer" check that the adjacent kq_mask inputs already use in these two functions. When the buffer is unallocated there is no data to upload, so skipping is correct. Fixes ggml-org#25191 Signed-off-by: liminfei-amd <[email protected]>
chat: trim messages sent to StepFun parser (fixes long reasoning loop… …s) (ggml-org#25238) * chat: trim messages sent to StepFun parser (fixes long reasoning loops) * add regression test; remove duplicate template * chat: trim StepFun content parts before rendering The StepFun trim workaround ran on the already-rendered messages, where typed content parts have been concatenated into a single string, so the per-part whitespace could no longer be reached. Move the trim ahead of rendering and apply it to content_parts text as well as the string content and reasoning_content. Adds a content-parts regression test. Co-Authored-By: Piotr Wilkin <[email protected]> Assisted-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: tarruda <[email protected]>
llama : add llama_model_ftype_name() (ggml-org#25134) * llama : add llama_model_ftype_name() Expose the model file type (quantization) name, e.g. "Q8_0" or "Q4_K - Medium", through a new public C API. The returned pointer is valid for the lifetime of the model and nullptr when the model is invalid or the file type is unknown. Signed-off-by: Adrien Gallouët <[email protected]> * Export enum Signed-off-by: Adrien Gallouët <[email protected]> * s/llama_model_ftype_name/llama_ftype_name/ Signed-off-by: Adrien Gallouët <[email protected]> * Move "(guessed)" to the front in llama_ftype_name Prepend the "(guessed)" label instead of appending it. This allows removing the non-thread-safe static std::string, making the function allocation-free. Signed-off-by: Adrien Gallouët <[email protected]> * Add LLAMA_FTYPE_PREFIX Signed-off-by: Adrien Gallouët <[email protected]> * Dont check for model Signed-off-by: Adrien Gallouët <[email protected]> --------- Signed-off-by: Adrien Gallouët <[email protected]>
opencl: allow loading precompiled binary kernels from library (ggml-o… …rg#23042) * opencl: allow loading binary kernel * opencl: add libdl.h * ggml-backend-dl is in ggml, which depends backend libs, thus ggml-opencl cannot depend on ggml-backend-dl * add libdl.h to break cyclic dep * opencl: allow loading bin kernel lib * opencl: load `gemm_moe_mxfp4_f32_ns` from kernel lib if available * opencl: load q8_0 gemm from kernel lib * opencl: load q4_0 moe gemm from kernel lib * opencl: load q4_1 moe gemm from kernel lib * opencl: load q4_k moe gemm from kernel lib * opencl: always declare `get_adreno_bin_kernel_func_t` * opencl: rephrase message * opencl: fix for rebase * opencl: update doc
cuda : prevent integer truncation and overflow errors when using KQ m… …ask strides in flash_attn_mask_to_KV_max kernel (ggml-org#24945) Co-authored-by: Stanisław Szymczyk <[email protected]>
Revert "sched : reintroduce less synchronizations during split compute ( ggml-org#20793)" (ggml-org#25138)
PreviousNext