Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[BUG] Concurrent send_and_wait calls with model specified in session triggers Failed to list models: 429 rate limiting #299

@msftnate

Description

@msftnate

Summary

When creating multiple concurrent sessions with a specific model specified in the session config, the SDK calls list_models on each send_and_wait() call rather than caching the result. This causes 429 rate limiting errors at moderate concurrency levels (~100 sessions).

Environment

  • SDK: GitHub Copilot Python SDK (copilot package)
  • Python: 3.11+
  • OS: Windows 11

Steps to Reproduce

import asyncio
import os

from copilot import CopilotClient

MODEL = "claude-opus-4.5"
NUM_SESSIONS = 100


async def create_session(client, index):
    # Step 1: Create session
    try:
        session = await client.create_session({"model": MODEL})
    except Exception as e:
        return False, f"[create_session] {e}"
    
    # Step 2: Send and wait
    try:
        await session.send_and_wait({"prompt": "Say hello"}, timeout=120)
    except Exception as e:
        return False, f"[send_and_wait] {e}"
    
    # Step 3: Cleanup
    try:
        await session.destroy()
    except:
        pass
    
    return True, None


async def main():
    client = CopilotClient()
    await client.start()
    
    print(f"Creating {NUM_SESSIONS} sessions with model={MODEL}...")
    
    results = await asyncio.gather(*[
        create_session(client, i) for i in range(NUM_SESSIONS)
    ])
    
    await client.stop()
    
    success = sum(1 for ok, _ in results if ok)
    failed = [err for ok, err in results if not ok]
    
    print(f"\nResults: {success}/{NUM_SESSIONS} succeeded")
    if failed:
        print(f"Errors ({len(failed)}):")
        for err in set(failed):
            print(f"  [{failed.count(err)}x] {err[:80]}")


if __name__ == "__main__":
    asyncio.run(main())

Output

Creating 100 sessions with model=claude-opus-4.5...

Results: 66/100 succeeded
Errors (34):
  [34x] [send_and_wait] Session error: Execution failed: Error: Failed to list models: 429

Potential Expected Behavior

  • Use a cached list of models after the first session.send_and_wait or bound the list of models upon client.create_session({"model": MODEL})

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions