@booxter for you

Confirming this works in my env (was failing to compile llama-cpp-python before). Thank you!

I wonder if this may actually break the CI. See here I get a failure in functional suite for a series with this patch included: https://github.com/instructlab/instructlab/actions/runs/9720826409/job/26832726682?pr=1275

  Aborted!
  Exception ignored on calling ctypes callback function: <function llama_log_callback at 0x7fe4988323b0>
  Traceback (most recent call last):
    File "/home/runner/work/instructlab/instructlab/.tox/py310-functional/lib/python3.10/site-packages/llama_cpp/_logger.py", line 23, in llama_log_callback
      @llama_cpp.llama_log_callback
  KeyboardInterrupt: 
  Error: ilab model chat command failed

Without this patch, tests pass. So non-native implementation seems to produce a traceback on KeyboardInterrupt that makes the test case fail.

Have you tried to run functional suite with the patch?

NOTE: I think this also reveals that tox.ini changes are not gated against. Though I see that tox.ini is explicitly listed in paths for different workflows (lint etc.) So I'm confused why no workflows triggered for this patch.

UPD: Maybe workflows are fine, at least I can get them triggered by just touching the file here: #1528

The old HEAD state triggered the actions: https://github.com/instructlab/instructlab/actions/runs/9716082740/job/26818894234 but not the one that was force-pushed afterwards. Was there some interruption in github workflow service that this PR caught?

The function tests are flaky and regularly fail without any apparent reason. Restarting failed tests usually fixes the problem. I had to restart test in multiple PRs in the past few days.

@tiran The problem is that they were not triggered on this PR at all. And for what I see in my series until I included this patch here (for testing) - it was not flaky, it was consistently failing on this exact test case and with this exact traceback I show above. I think we should get fresh functional result here before considering merging it.

#1526 includes this patch and tests are passing. Strange...

The improvement landed with #1526 .