it seems I can't copy build-info.h out of the source directory since it's copied to the nix store and made read-only before the build is run. Would it make sense to output build-info.h to CMAKE_CURRENT_BUILD_DIR instead of CMAKE_CURRENT_SOURCE_DIR?

I appologize for the small random commit, I'm having to make changes in github.dev before committing them and testing them on my local machine since github doesn't seem to let you commit from non-codespace environments anymore.

I appologize for the small random commit

It shouldn't matter because it would be squashed together when it's merged, but maybe you want to turn this into a draft if it's "WIP"?

Nice fix, you will also need to cp $src/ggml-metal.metal $out/bin if you want to store layers in VRAM using -ngl using Metal on aarch64-darwin.

@swdunlop thanks, I don't develop on a mac so I didn't realize that's required.

With that change, I can work on C++ projects using llama.cpp with Nix on MacOS using the following flake:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/master";
    flake-utils.url = "github:numtide/flake-utils";
    llama-cpp.url = "github:swdunlop/llama.cpp";
    # llama-cpp.url = "github:ggerganov/llama.cpp";
  };
  outputs = { self, nixpkgs, flake-utils, llama-cpp }: flake-utils.lib.eachDefaultSystem (system:
    let 
      pkgs = nixpkgs.legacyPackages.${system}; 
      llama = llama-cpp.packages."${system}".default;
    in {
      devShells.default = pkgs.stdenv.mkDerivation {
        name = "llama-dev-shell";
        buildInputs = [ pkgs.clang-tools llama ];
        shellHook = ''
          PATH="${pkgs.clang-tools}/bin:$PATH"
          C_INCLUDE_PATH="${llama}/include"
        '';
      };
    }
  );
}

(The shellHook is necessary to make clang and clangd happy with Nix.)

Superseded by #2570

Superseded by #2570

This looks like a typo. This is #2570. I'm guessing you meant #3159

@ggerganov #3159 seems to ignore some of the important headers as more then just llama.h is required for full functionality (from my understanding). Is there a reason why you've chosen to to only include llama.h?

you are right, llama.h includes ggml.h (and ggml-cuda.h if building for cuda).
but the others are not necessary for the llama.cpp api.

Wouldn't it still lead to include errors if those headers are not preset? I'm mainly asking if there is a reason to re-open this PR since the other one doesn't seem complete.

Yea, please reopen, but with only the necessary headers.