I am unsure of why such a thing occurs. I'm passing in the json.gbnf grammar to restrict the output. I have also set the following parameters -

• seed: 123
• temperature: 0.0
• n_predict: 512
• typical_p: 0.9

Here is an example prompt I pass - <|user|>You are JARVIS, a capable and knowledgable AI assistant. You only respond in JSON.<|end|>\n<|user|>Name top 5 things to eat in Chicago. Also explain why.<|end|><|assistant|><s>

And here is the response I receive -
{\n "top_5_food": [\n {\n "name": "Chicago-style Deep Dish Pizza",\n "reason": "This iconic dish is a must-try for its unique crust and generous toppings, originating from the city itself."\n },\n {\n "name": "Hot Dog on a Poppy Bun",\n "reason": "A Chicago classic featuring a poppy seed bun with all traditional toppings like mustard, relish, pickle, onions and sport peppers."\n },\n {\n "name": "Cotton Candy",\n "reason": "An indulgent treat that\'s perfect for kids or anyone looking for a sweet, fluffy cloud of sugar."\n },\n {\n "name": "Chicago-style Ribeye Sandwich",\n "reason": "A juicy steak sandwich with crispy fries and coleslaw on the side. It\'s known for its generous portions and rich flavors."\n },\n {\n "name": "Garrett Popcorn",\n "reason": "This unique popcorn is a Chicago specialty, made with caramelized sugar and butter, giving it a sweet and salty taste that\'s hard to resist."\n }\n ]\n}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Is this because of how n_predict works internally or is it model related?