Adds more stop-tokens to chat formatter #699
Conversation
cli/server.py
Outdated
There was a problem hiding this comment.
can we change this to
chatFormatterResponse.stop.extend(["<|system|>", "<|user|>", "<|assistant|>"])
(with properly formatting to make linter happy)
i think this reads better as we are actually adding more EOS tokens to the existing list which is set to be only the eos_token in the Jinja2ChatFormatter initialization.
There was a problem hiding this comment.
Yeah sure, will update
hickeyma
left a comment
There was a problem hiding this comment.
@JamesKunstle Just wondering if this is covered by the work done in #559 and #547?
|
this is to fix #159 |
Signed-off-by: James Kunstle <[email protected]>
f15b0e9 to
0bee5cf
Compare
|
@xukai92 functional test caught a problem- I misinterpreted the return from -> can close this PR without merging, it doesn't fix the problem. |
|
Closing as per @JamesKunstle comment in #699 (comment) |
We need to stale things! --------- Signed-off-by: JJ Asghar <[email protected]> Co-authored-by: BJ Hargrave <[email protected]>
Resolves #452
May fix #159, #137
Adds more stop-tokens to
ChatFormatterobject:[<|endoftext|>, <|system|>, <|user|>, <|assistant|>]