Description
Name and Version
llama_cpp_python==0.2.88
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
Other (Please specify in the next section)
Issue at /llama-cpp-python_16ea09f94c0346afa022d988e7934741/vendor/llama.cpp/src/llama.cpp:14775 due to libggml.so
Command line
Problem description & steps to reproduce
I have a flask application running in a docker container instance on a VM. It takes 'query' as input and uses an underlying RAG system to answer the query, and returns a json response containing the LLM output.
I use LlamaCppEmbeddings (which has a dependency on llama_cpp_python) to load a nomic-embed-text-v1.5.Q8_0.gguf embedding model, it is used to perform dense vector search in the knowledge base. That vector index was also created using the same embedding model, and stored in a FAISS db ~50MB (index.faiss+index.pkl), which is loaded in runtime. I use Llama3.3-70B LLM model for the generation based on top-4 cosine similarity matches. The vector index and embedding model are mounted as docker volumes, whereas the LLM is used as an API.
Now if I make >3-4 simultaenous API requests to the endpoint at which the container is running, then I get a 139 exit code on the container. And the following error messages in the docker logs:
First Bad Commit
No response
Relevant log output
2025-06-27T10:45:59.802448067Z /tmp/pip-install-kfbxdcj6/llama-cpp-python_16ea09f94c0346afa022d988e7934741/vendor/llama.cpp/src/llama.cpp:14775: GGML_ASSERT(backend_embd != nullptr) failed
2025-06-27T10:45:59.837457263Z /usr/local/lib/python3.11/site-packages/llama_cpp/lib/libggml.so(+0xea48)[0x7f3f1e6daa48]
2025-06-27T10:45:59.837457263Z /usr/local/lib/python3.11/site-packages/llama_cpp/lib/libggml.so(ggml_abort+0x135)[0x7f3f1e6dc715]
2025-06-27T10:45:59.837530429Z /usr/local/lib/python3.11/site-packages/llama_cpp/lib/libllama.so(llama_decode+0x175e)[0x7f3f1e8acf0e]
2025-06-27T10:45:59.837568832Z /lib/x86_64-linux-gnu/libffi.so.8(+0x6f7a)[0x7f3f2ba60f7a2025-06-27T10:45:59.837605963Z ]
2025-06-27T10:45:59.837605963Z /lib/x86_64-linux-gnu/libffi.so.8(+0x640e)[0x2025-06-27T10:45:59.837642401Z 7f3f2ba6040e]
2025-06-27T10:45:59.837642401Z /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0xcd)2025-06-27T10:45:59.837679221Z [0x7f3f2ba60b0d]
2025-06-27T10:45:59.837714084Z /usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so(+0x10c5f)[0x7f3f2a86bc5f]
2025-06-27T10:45:59.837764411Z /usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so(+0x7c3f)[0x7f3f2a862c3f]
2025-06-27T10:45:59.837848156Z /usr/local/bin/../lib/libpython3.11.so.1.0(_PyObject_MakeTpCall+0x6f)[0x7f3f2cdddf8f]
2025-06-27T10:45:59.837970557Z /usr/local/bin/../lib/libpython3.11.so.1.0(_PyEval_EvalFrameDefault+0x66d)[0x7f3f2cde6a0d]
2025-06-27T10:45:59.838008796Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x1b9c1a)[0x7f3f2cde2c1a]
2025-06-27T10:45:59.838070588Z /usr/local/bin/../lib/libpython3.11.so.1.0(_PyEval_EvalFrameDefault+0x43ab)[0x7f3f2cdea74b]
2025-06-27T10:45:59.838211593Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x1b9c1a)[0x7f3f2cde2c1a]
2025-06-27T10:45:59.838211593Z /usr/local/bin/../lib/libpython3.11.so.1.0(_PyEval_EvalFrameDefault+0x43ab)[0x7f3f2cdea74b]
2025-06-27T10:45:59.838211593Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x1b9c1a)[0x7f3f2cde2c1a]
2025-06-27T10:45:59.838287570Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x1f2f10)[0x7f3f2ce1bf10]
2025-06-27T10:45:59.838362371Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x2a0694)[0x7f3f2cec9694]
2025-06-27T10:45:59.838434099Z /usr/local/bin/../lib/libpython3.11.so.1.0(+0x26f5d4)[0x7f3f2ce985d4]
2025-06-27T10:45:59.838521901Z /lib/x86_64-linux-gnu/libc.so.6(+0x891c4)[0x7f3f2cacb1c4]
2025-06-27T10:45:59.838627631Z /lib/x86_64-linux-gnu/libc.so.6(+0x10985c)[0x7f3f2cb4b85c]