Walkthrough

Replaced use of max_tokens with a conditional max_completion_tokens in inference calls: when serverConfig.inference.useMaxCompletionTokens is true the code emits max_completion_tokens = serverConfig.inference.maxOutputTokens; otherwise it continues to emit max_tokens = serverConfig.inference.maxOutputTokens. Control flow and response_format handling unchanged.

Changes

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Inference as inference.ts
    participant OpenAI

    Client->>Inference: request inferFromText / inferFromImage
    Note right of Inference #DDEBF7: Read serverConfig.inference\n(useMaxCompletionTokens, maxOutputTokens)
    alt useMaxCompletionTokens == true
        Inference->>OpenAI: send completion with\n`max_completion_tokens = maxOutputTokens`
    else useMaxCompletionTokens == false
        Inference->>OpenAI: send completion with\n`max_tokens = maxOutputTokens`
    end
    OpenAI-->>Inference: completion response
    Inference-->>Client: return response

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Pre-merge checks and finishing touches

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Can this be looked at and hopefully merged? It would be really nice to get this merged so I can use gpt-5-mini for summarization instead of being stuck on gpt-4.1-mini, especially since gpt-5-mini is only a bit over half the price for input tokens.

Is there anything blocking this fix? I updated my instance to use gpt-5-nano after adding more API credits, and I'd prefer to leave it on gpt-5-nano, but I really like the AI summarization and tagging features.

hey folks, sorry for taking too long to get back to this. Biggest concern with this PR is what it'll break. People use the openai client for almost all other providers, and I don't know which providers wouldn't support this. So here's my suggestion. Can we add a new env variable that controls which field to pass, and default it as the old behavior for now, and then in the next release we switch the default after giving people a headsup in the release notes?

This makes sense to me. Might be best to just leave a legacy_max_tokens flag for the foreseeable, since surely a lot of other projects aren't going to rush to update just to match this against OpenAI.

@MohamedBassem Took a crack at an update - decided to go with a variable that controls the field to pass vs having 2 token fields for now as that seemed a bit clearer to me, and less risk of someone setting both or similar and having to explain what happens in those cases.

Thanks a lot @BenjaminMichaelis, and sorry once more for the delay :) This will be in the nightly image in around 30mins.

When should we look for this in the regular release? I've added the INFERENCE_USE_MAX_COMPLETION_TOKENS env variable and set it to true, but I still get "Something went wrong" when attempting to Summarize a bookmark. I re-pulled images after making the change, so I'm assuming it's only on the nightly builds and that's why I'm still getting an error.

Should be in the next release