Cache structs.fields() result for non-generic structs - #1005
Conversation
Store the computed tuple of FieldInfo on the class as __struct_field_info__ to avoid expensive get_class_annotations() and FieldInfo recreation on every call. Generic aliases are not cached since type parameters affect annotations. Closes #963
|
|
||
| # Cache for non-generic structs | ||
| if not is_generic: | ||
| cls.__struct_field_info__ = result |
There was a problem hiding this comment.
Mutating the type as a result of inspection seems wrong to me. Is there no other way we can cache or otherwise close the gap in performance here? Also, does this work if the Struct is configured as frozen?
There was a problem hiding this comment.
Thanks for the review!
Let me address your points in order.
Frozen structs. Yes, this works. frozen=True sets tp_setattro = Struct_setattro_frozen on the struct class itself, which only blocks attribute assignment on instances. The cache write cls.__struct_field_info__ = result goes through type(cls).tp_setattro (i.e. StructMetaType's slot, inherited from PyType_Type), which isn't overridden and isn't affected by frozen. That said, you're right that it's not obvious from the diff, I just pushed an explicit test for frozen structs to make the guarantee visible.
Why mutation, and what alternatives I considered. The dominant cost in fields() is _get_class_annotations() (typing.get_type_hints()), which is what makes the difference between 0.44s and 7.5s on the benchmark. I looked at a few alternatives before landing on __struct_field_info__:
- Module-level
WeakKeyDictionary: keepsfields()non-mutating, but adds an extra lookup on every call and another global to reason about. Felt strictly worse. - Eager compute in
StructMeta_new: fastest in theory, but breaks on forward references, which is exactly why the current implementation is lazy. - Caching only
hintsinstead of the full tuple: still requires mutating the class, and we'd still rebuildFieldInfoobjects on every call (a meaningful chunk of the remaining cost).
In the end I picked the option with the smallest surface and the biggest win, treating __struct_field_info__ as a conventional cache slot (similar to how CPython uses dunder names for lazily-populated metadata).
On "mutation feels wrong". I hear you and I agree it's not ideal aesthetically. I see two ways forward:
- Keep the current approach, document
__struct_field_info__as an internal cache slot, and add the frozen test. Pragmatic, minimal diff. - Port
fields()to C as a follow-up PR.StructMetaObjectalready holds everything we need (struct_encode_fields,struct_offsets,struct_types,struct_defaults), so for non-generic structs we can skiptyping.get_type_hintsentirely and build theFieldInfotuple directly. This would likely beat even the cached Python version and fully sidesteps the mutation question.
Personally I lean toward option 2 (port to C): it gives the best perf, removes the class mutation, and fits naturally into the existing StructMetaObject C infrastructure. If you're on board, I'm happy to close this PR and open a new one with the C port. @jcrist, would love to hear your take as well, since this touches StructMeta design.
There was a problem hiding this comment.
Good call on the mutation. I've reworked it: instead of writing __struct_field_info__ onto the class, the cache now lives in a module-level weakref.WeakKeyDictionary keyed on the struct type. The class is no longer touched at all, so inspection stays side-effect free, nothing shows up in dir()/vars(), and there's no risk of a subclass picking up a parent's cached tuple via MRO. Entries drop automatically when a class is collected (verified with dynamically-created defstruct classes). The frozen question is moot now since we never set an attribute on the instance or the class.
On the C-port idea I floated earlier: I dug into it and it doesn't pan out. FieldInfo.type has to return the resolved Python annotation (e.g. int | None, Annotated[int, Meta(...)], forward refs resolved). StructMetaObject doesn't store that, only the compiled TypeNode, which is lossy (e.g. Meta(gt=3) and Meta(ge=4) collapse to the same node). Rebuilding the annotation from a TypeNode would mean reimplementing typing.get_type_hints in C, and it would still need lazy resolution for forward refs. So I'm dropping that option; the side-table cache gives the same speedup without the mutation.
|
@ofek @provinzkraut would appreciate your take here. The blocker: this PR caches Two paths:
Leaning toward 2 but wanted a second opinion before committing. |
|
The benchmarks didn't run (#1091) so I have to update the branch once more. Sorry for the noise! |
Merging this PR will improve performance by ×3.4
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | test_fields[arm-complex_class] |
119.2 µs | 17.4 µs | ×6.8 |
| ⚡ | Simulation | test_fields[arm-complex_instance] |
118.6 µs | 19.4 µs | ×6.1 |
| ⚡ | Simulation | test_fields[arm-class] |
104.2 µs | 17.6 µs | ×5.9 |
| ⚡ | Simulation | test_fields[arm-instance] |
104.9 µs | 19.5 µs | ×5.4 |
| ⚡ | Simulation | test_decode_type[arm-msgpack-None] |
760.6 µs | 682.7 µs | +11.4% |
| ⚡ | Simulation | test_pickle_load[arm] |
1.8 ms | 1.6 ms | +11.16% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing 963-cache-fields (c86ba9e) with main (f51f378)
Footnotes
-
135 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
re benchmarks: Since this is aimed to improved performance, we should probably also have a benchmark that reflects this. Would also prevent future regressions. I've put up #1092 with some more benchmarks |
@Siyet as I mentioned here, I believe we should do both; We don't have to recompute this, making it lazy and storing is is the right way. However, we also already have all the information internally, so it would make more sense to expose that from the C-side. I'd like to see how expensive that is performance wise. Maybe caching isn't needed at all. |
|
@provinzkraut I've measured the C-side option. The catch: the C side doesn't actually retain the resolved annotations.
So a C-side
A C port without caching is therefore at best ~1.5x. For C to beat the cache it would have to store the resolved annotations on |
|
Merged current main in, no content change. @ofek the class mutation you flagged in April is gone as of d6919fe. The cache is a module-level @provinzkraut the C-side question is answered above with measurements: One thing I want your call on before this goes in. |
Reopening #997 (auto-closed when fork was deleted).
Summary
fields()result tuple on the struct class as__struct_field_info__get_class_annotations()call andFieldInfoobject recreation on every invocationMyStruct[int]) are not cached since type parameters affect annotationsCloses #963
Benchmark (1M calls, 3 fields)
dataclasses.fields()msgspec.structs.fields()