@esnible thanks for pointing this out. I thought we are calling ollama through the LiteLLM interface, which should make the requests OpenAPI compatible. However, I didn't find any example documentation regarding stop_sequences specifically on LiteLLM's Ollama docs page. But I'm also wondering if we need to add temperature: 0 to the example PDL programs. The current PDL docs say that it is the case with Granite models. I'm not sure if it is recognizing it's a Granite model.

@jgchn I also have not found documentation for the differences in parameter passing between OpenAPI, Ollama, and Replicate.

So far, the only way I can detect when LiteLLM passing something to Ollama that it doesn't expect is to watch Ollama's logs. It is possible that Ollama returns a warning through the REST interface, which could be caught by a LiteLLM callback, but I haven't found that yet.

I strongly recommend setting the temperature to 0 for every example, because we are testing the examples and would like the testing to be as reliable as possible. I have a PR to skip the tests that are currently failing (#538 ). Once this PR goes in, my hope is that we can resume testing more examples. My hope is to have nearly all examples tested.

@jgchn I introduced a pattern so that we can have default parameters for each model. We have no defaults yet for Ollama/granite-code!!!

My feeling is when we identify good defaults for a model we should add them at https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl_ast.py#L712 so that users do not need to know the best defaults for each model.

Ah nice I wish these defaults were in the docs!