I am running into this issue but with ./main:

./main -m models/Phi-3-mini-4k-instruct-gguf/Phi-3-mini-4k-instruct-q4.gguf -n 1024 -e -c 4096 -ngl -1 -r "['<|end|>']" -p '1-2='

1-2=0<|end|><|assistant|> The equation 1-2=0 is incorrect. The correct result of 1-2 is -1. Therefore, the equation 1-2 ≠ 0 is true.<|end|><|assistant|> Yes, that is correct. The equation 1-2 does not equal 0; instead, it equals -1. The equation 1-2 = 0 is false.<|end|><|assistant|> Absolutely, you've got it right. The expression 1-2 indeed equals -1, not 0. So, the equation 1-2 = 0 is not true.<|end|><|endoftext|> [end of text]

Commit: 8a56075

On commit 928e0b7 I get gibberish which is probably related to #6944

It looks like -r is not being handled correctly even if added manually?

Another example, using the chat template:

./main -m models/Phi-3-mini-4k-instruct-gguf/Phi-3-mini-4k-instruct-fp16.gguf -n 256 -e -c 4096 -ngl -1 -r "[<|end|>]" -p '<|user|>Result of 1-2:<|end|>\n<|assistant|>'

<|user|>Tell me the final result of 1-2:<|end|>
<|assistant|> The result of 1 - 2 is -1.<|end|><|assistant|> When you subtract 2 from 1, you get -1. Here's the calculation:

1 - 2 = -1

Subtraction is the operation of removing one quantity from another. So if you have 1 item and remove 2 items, you cannot do so directly; instead, you end up with a deficit, represented by -1 in this context.<|end|><|endoftext|> [end of text]

Or am I doing something wrong here?

server: phi-3 end token not handled? #6903

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions