Replaying an AgentChat run with the model server switched off (verified on autogen-agentchat 0.7.5) #8217

Verified against autogen-agentchat 0.7.5 today: an AgentChat run can be recorded from outside the process and then re-run with no model server present.

Disclosure: I work on the tool (OrcaReplay, Apache-2.0, no paid tier).

Why AutoGen needs no special support

OpenAIChatCompletionClient with no base_url argument falls through to the OpenAI SDK's own OPENAI_BASE_URL. I checked rather than assumed:

client = OpenAIChatCompletionClient(model="qwen3.5:0.8b", model_info={...})
print(client._client.base_url)
# → http://localhost:11434/v1/

So a recording proxy only has to set that variable for the child process. Nothing is installed into AutoGen — no callback, no wrapper, no change to your agent code.

The run

$ orca record generic-openai -- python autogen_agent.py
autogen said: AUTOGEN
info recorded run=run_59d3f1a10592 events=6 blobs=1 exit=0

$ taskkill /F /IM ollama.exe          # model server gone
$ curl -m 3 http://localhost:11434/v1/models    # nothing listening

$ orca replay run_59d3f1a10592
autogen said: AUTOGEN
info replay.done reused=1/1 exact=1 divergences=0 unmatched=0 exit=0

A real AssistantAgent with agent.run(task=…), not a bare SDK call.

What it is useful for

Re-running an agent while you change the system message, the tools, or the termination condition — the model responses stay fixed, so you see what your change did rather than what the sampler did. Also useful in CI, and for chasing one bad answer without paying for it each time.

From a checkpoint the run can instead continue live on a different model, which is the honest way to compare two models on the same task.

Two limits worth stating

• It captures the HTTP going to the model, not AutoGen's internals — no message-passing graph, no per-agent spans. Not a substitute for tracing; a different question.
• egress=blocked on replay means model-provider egress. Recorded tool calls still execute for real, so a recorded shell or HTTP tool runs again. It is not a sandbox.

Happy to be told OpenAIChatCompletionClient has other origin paths I have not exercised — I would rather fix the assumption than ship it.