You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
LlamaGrammar should not print its grammar rules when verbose=False
Current Behavior
The grammar rules are printed on each inference call, even when verbose=False is set in both the Llama() and LlamaGrammar.from_string() initializers
Environment and Context
Physical (or virtual) hardware you are using, e.g. for Linux: M3 Macbook Pro
Operating System, e.g. for Linux: MacOS
SDK version, e.g. for Linux:
$ python3 --version
Python 3.12.4
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
$ g++ --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Failure Information (for bugs)
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
Steps to Reproduce
The following is a minimum reproducible example
importllama_cppfromllama_cpp.llama_grammarimportJSON_GBNFdefmain():
llama=llama_cpp.Llama("/Users/joshl/Projects/MechanisticLLM/data/models/language/mistral-7b-instruct-v0.3-q4_k_m.gguf", verbose=False)
grammar=llama_cpp.LlamaGrammar.from_string(JSON_GBNF, verbose=False)
response=llama("What is the answer to life, the universe, and everything?")
print(response)
if__name__=='__main__':
main()
Because I've set LlamaGrammer.from_string(verbose=False), I expected that the JSON grammar would not be printed to the screen
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
LlamaGrammar should not print its grammar rules when
verbose=False
Current Behavior
The grammar rules are printed on each inference call, even when
verbose=False
is set in both theLlama()
andLlamaGrammar.from_string()
initializersEnvironment and Context
Physical (or virtual) hardware you are using, e.g. for Linux: M3 Macbook Pro
Operating System, e.g. for Linux: MacOS
SDK version, e.g. for Linux:
Failure Information (for bugs)
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
Steps to Reproduce
The following is a minimum reproducible example
Because I've set
LlamaGrammer.from_string(verbose=False)
, I expected that the JSON grammar would not be printed to the screenFailure Logs
The text was updated successfully, but these errors were encountered: