Thanks to visit codestin.com
Credit goes to github.com

Skip to content

♻️ Refactor internals to preserve APIRouter and APIRoute instances#15745

Merged
tiangolo merged 4 commits into
masterfrom
routers
Jun 14, 2026
Merged

♻️ Refactor internals to preserve APIRouter and APIRoute instances#15745
tiangolo merged 4 commits into
masterfrom
routers

Conversation

@tiangolo

@tiangolo tiangolo commented Jun 14, 2026

Copy link
Copy Markdown
Member

Pull Request

♻️ Refactor internals to preserve APIRouter and APIRoute instances

Supersedes #4794

Unblocks ✨ SO MANY THINGS ✨

Before this, router.include_router(other_router) would take each path operation from other_router and "clone" it, or recreate it from scratch.

This would mean that in the end there was only one top level router, part of the app.

The way it is structured here is that there are a few additional classes to handle intermediate metadata for router and route inclusion. That way the information of "router X includes Y and Y includes Z" is stored somewhere, without affecting (recreating / clonning) the final route.

Non Objective

Dependencies for 404

Originally in the other PR I intended to support dependencies that would be executed even for 404, but that would conflict with the fact that a router could not find a match, but the next router did find a match. Executing dependencies in the router that did not find a match would not make sense, they could consume the request, body, etc.

This original idea was discarded.

Breaking Change

Now router.routes is no longer a plain list of APIRoute objects, it can contain these intermediate objects that can contain additional routers, forming a tree.

Any logic that depended on iterating on the router.routes directly would be affected, that logic cannot expect to be able to extract data from a plain list of routes, as it's no longer a plain list but a tree.

Additionally, any logic that iterated on router.routes to modify them would now also see these new objects, and would not see all the routes in the app.

router.routes should be considered an internal implementation detail, only passed around to the FastAPI functions that need it.

Features

  • Adding routes after a router is included now works, they are reflected as they are not copied.
  • Including subrouter in mainrouter can be done before adding routes (path operations) to subrouter, because now the the entire object is stored instead of copying the routes.
  • As routes are not copied, in some cases that might save some memory.

Alpha Features

This is not documented yet, so it's not officially supported yet and could change in the future.

But, as APIRoute and APIRouter instances are now preserved, they could be customized.

APIRouter has two new methods, .matches() and .handle(), counterpart to the existing ones in APIRoute. With this a router could customize how it matches and handles requests. For example, it could match only requests that include some specific header, for example for handling versions in headers.

Still, for now, consider this very experimental and potentially changing and breaking in the future.

Future Features Enabled

  • Custom APIRoute subclasses (as desccribed above)
  • Custom APIRouter subclasses (as described above)
  • Dependencies per router
  • Exception handlers per router
  • Middleware per router
  • Other features planned

Discussion:

Description

AI Disclaimer

Codex with GPT 5.5, through a lot of planning and research iterations, for weeks, then the same for the implementation, with too many iterations to count to clean up the implementation, what's supported and not, docs, types, etc.

All code and tests manually reviewed by hand.

AI transcript

Checklist

  • This PR is an obvious typo fix, or it links to a GitHub Discussion for the proposed code change.
  • I added tests for the change.
  • The new or updated tests fail on the main branch and pass on this PR.
  • Coverage stays at 100%.
  • The documentation explains the change if needed.

@codspeed-hq

codspeed-hq Bot commented Jun 14, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 20 untouched benchmarks


Comparing routers (1e70f7c) with master (e2fcd55)1

Open in CodSpeed

Footnotes

  1. No successful run was found on master (016ab76) during the generation of this report, so e2fcd55 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@tiangolo tiangolo marked this pull request as ready for review June 14, 2026 12:32
@tiangolo tiangolo merged commit 8e1d774 into master Jun 14, 2026
49 checks passed
@tiangolo tiangolo deleted the routers branch June 14, 2026 12:35
@tiangolo tiangolo mentioned this pull request Jun 14, 2026
25 tasks
zhuangqh pushed a commit to kaito-project/kaito that referenced this pull request Jun 16, 2026
…2s_v3 SKU in E2E (#2104)

## What type of PR is this?
/kind bug

## What this PR does / why we need it

### 1. Replace `Standard_NC12s_v3` with `Standard_NV72ads_A10_v5` in E2E
tests

`Standard_NC12s_v3` (V100) is not available in `swedencentral` (and many
other regions), causing E2E webhook tests to fail with
`VMSizeNotSupported`.

`Standard_NV72ads_A10_v5` (2×A10 = 48Gi GPU memory) is used because:
- It matches the "2gpu" semantics of the webhook tests
- It fits phi-4 on a single node (no multi-node Transformers issue)
- It is widely available in the regions where E2E runs

### 2. Fix vLLM inference server crash caused by
`prometheus-fastapi-instrumentator` incompatibility

**Root cause of E2E `validateInferenceResource` timeouts:**

FastAPI 0.137.0
([fastapi/fastapi#15745](fastapi/fastapi#15745))
refactored `include_router()` to wrap routers in `_IncludedRouter`
objects that lack a `.path` attribute. This breaks
`prometheus-fastapi-instrumentator` middleware
([trallnag/prometheus-fastapi-instrumentator#370](trallnag/prometheus-fastapi-instrumentator#370)),
causing **every HTTP request** to the vLLM inference server to return
500:

```
AttributeError: '_IncludedRouter' object has no attribute 'path'
```

vLLM starts and loads models successfully, but all health checks and
inference requests crash in the Prometheus middleware before reaching
the actual handler. This makes the workspace permanently "not ready"
regardless of timeout duration.

**Fix:** Pin `fastapi[standard] <0.137.0` in requirements.txt until the
upstream instrumentator is fixed.

### 3. Revert timeout bump (30m → 20m)

The previous commit bumped `validateInferenceResource` timeout from 20m
to 30m, but that was never the real issue — the middleware crash means
no amount of waiting helps. Reverted to original 20m.

## Changes
- `test/e2e/webhook_test.go`: replace instanceType in webhook validation
tests
- `presets/workspace/dependencies/requirements.txt`: pin
`fastapi<0.137.0`
- `test/e2e/preset_test.go`: revert validateInferenceResource timeout to
20m
georgi-smasint pushed a commit to supermassive-intelligence/scalarlm that referenced this pull request Jun 17, 2026
FastAPI 0.137.0 (fastapi/fastapi#15745) keeps _IncludedRouter wrappers in
app.routes; prometheus-fastapi-instrumentator reads route.path unconditionally
-> AttributeError -> HTTP 500 on every vLLM endpoint. cray's health check
asserts vLLM /health == 200, so the stack reports vllm down and the finetune
sweep fails with RESTART_FAILED. fastapi-utils pulls FastAPI transitively, so
the ceiling lives in requirements-vllm.txt.

See vllm-project/vllm#45597, trallnag/prometheus-fastapi-instrumentator#370.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant