Re:

• [Bug]: FastAPI 0.113.0 breaks vLLM OpenAPI vllm-project/vllm#8212
• [Bug]: when curl /chat/completions, TypeError: Unable to evaluate type annotation 'Required[Union[str, Iterable[ChatCompletionContentPartTextParam]]]'. vllm-project/vllm#8450
• Request Types of TypedDict cannot be parsed for Swagger / OpenAPI Specifications when Required attribute is used. #11952

(and potentially others)

I think I found the underlying cause. We're still supporting pydantic.Required in FastAPI, which has the possibility to shadow the new Required type. To me, this seems like a Pydantic + fastapi bug. Here's how 0.113.0 introduced it:

1. fastapi has the new get_model_fields() function, which deals with TypeAdapter(), and is called for every field
2. TypeAdapter gets a globalns from the sys._getframe(), meaning it gets the ns from _compat. This is important because pydantic bases their type evaluations from that globalns. Why they are doing this is a question for later.
3. Because Required = PydanticUndefined is in _compat.py, we are getting that value set in the globalns
4. Independently, openai uses TypedDict with the Required annotations in its fields.
5. vllm (and potentially other libs) uses fastapi + pydantic + openai TypedDict classes

The smoking gun was: TypeError: 'pydantic_core._pydantic_core.PydanticUndefinedType' object is not subscriptable

Focusing on vllm-project/vllm#8450, the offending type annotation was Required[Union[str, Iterable[X]]. It needed to have been Required that was a PydanticUndefinedType, otherwise it would be complaining about Union[str, Iterable[X]], or some other annotation. I ran the snippet here using fastapi 0.114.1, pydantic 2.9.0, vllm 0.6.1, python = 3.8.20, and was able to reproduce the issue again. Using pdb.pm(), I was able to inspect the globalns and connect the dots.

How to test

Run the code example in this comment Python 3.8. It will fail without the changes, and pass with the changes.