Code

add KV cache quantization options#1307

Code

Merged

abetlen merged 5 commits intoabetlen:mainabetlen/llama-cpp-python:mainfrom

Limour-dev:mainLimour-dev/llama-cpp-python:mainCopy head branch name to clipboard

Apr 1, 2024

Commits

Commits on Mar 28, 2024

• add KV cache quantization options

  Show description for 94a9519
  Limour-dev

  authored
  94a9519

  Copy full SHA for 94a9519

Commits on Mar 31, 2024

• Merge branch 'main' into Limour-dev/main

  abetlen

  committed
  c4a2f85

  Copy full SHA for c4a2f85

• Add ggml_type

  abetlen

  committed
  1a6a9a3

  Copy full SHA for 1a6a9a3

• Use ggml_type instead of string for quantization

  abetlen

  committed
  fcb8051

  Copy full SHA for fcb8051

Commits on Apr 1, 2024

• Add server support

  abetlen

  committed
  7828382

  Copy full SHA for 7828382