I have run quantization on ggml-small.en.bin to produce ggml-small.en.bin-q5_0.bin
Quantization proceeded without any errors.
When I run the model, it attempts to load, but throws the following:
GGML_ASSERT: ggml.c:4288: wtype != GGML_TYPE_COUNT
Abort trap: 6
Same issue occurs when I try 8-bit quantization.
Running on a 2015 Macbook Air, if this is any use.
I have run quantization on ggml-small.en.bin to produce ggml-small.en.bin-q5_0.bin
Quantization proceeded without any errors.
When I run the model, it attempts to load, but throws the following:
GGML_ASSERT: ggml.c:4288: wtype != GGML_TYPE_COUNT
Abort trap: 6
Same issue occurs when I try 8-bit quantization.
Running on a 2015 Macbook Air, if this is any use.