refs: replace go_generics with Go generics #12470

tamird · 2026-01-09T14:43:53Z

go_stateify: add generics support
ilist: replace go_generics with Go generics
refs: replace go_generics with Go generics

There are many more to do, but these are the most often used.

tamird · 2026-01-09T15:15:31Z

@avagin @ayushr2 let me know if you'd prefer this in separate PRs.

avagin · 2026-01-09T15:50:50Z

@nixprime could you take a loot at this pr too?

tamird · 2026-01-09T16:18:14Z

@avagin could you import this so that tests run? I haven't run all the tests yet (still need #12461).

ayushr2 · 2026-01-09T18:24:16Z

@tamird Have we benchmarked the new ilist or refs implementation compared to their old go_generics one? IIRC, with Go generics, we add interface method calls and these types are on the hot paths for various subsystems.

nixprime

Unlike go_generics, native Go generics only instantiate distinct code for different argument "shapes"; see https://github.com/golang/proposal/blob/master/design/generics-implementation-gcshape.md. AFAIU, all pointers have the same shape, so all instances of ilist.Entry[*T] and ilist.List[*T] use the same generated code, necessitating dynamic dispatch for ilist.Linker method calls after this change.

For example, before this change:

TEXT gvisor.dev/gvisor/pkg/sentry/kernel.(*pendingSignals).enqueue(SB) pkg/sentry/kernel/pending_signals.go
...
  pending_signals_list.go:112   0xa9ec71                90                      NOPL
  pending_signals_list.go:231   0xa9ec72                48c70000000000          MOVQ $0x0, 0(AX)
  pending_signals_list.go:113   0xa9ec79                488b4c2410              MOVQ 0x10(SP), CX
  pending_signals_list.go:113   0xa9ec7e                488b542430              MOVQ 0x30(SP), DX
  pending_signals_list.go:113   0xa9ec83                488b5cca08              MOVQ 0x8(DX)(CX*8), BX
  pending_signals_list.go:238   0xa9ec88                833d312a7d0100          CMPL runtime.writeBarrier(SB), $0x0
  pending_signals_list.go:238   0xa9ec8f                7408                    JE 0xa9ec99
  pending_signals_list.go:238   0xa9ec91                e80aa59eff              CALL runtime.gcWriteBarrier1(SB)
  pending_signals_list.go:238   0xa9ec96                49891b                  MOVQ BX, 0(R11)
  pending_signals_list.go:238   0xa9ec99                48895808                MOVQ BX, 0x8(AX)
  pending_signals_list.go:114   0xa9ec9d                488b5cca08              MOVQ 0x8(DX)(CX*8), BX
  pending_signals_list.go:114   0xa9eca2                4885db                  TESTQ BX, BX
  pending_signals_list.go:114   0xa9eca5                741d                    JE 0xa9ecc4
  pending_signals_list.go:231   0xa9eca7                833d122a7d0100          CMPL runtime.writeBarrier(SB), $0x0
  pending_signals_list.go:231   0xa9ecae                740f                    JE 0xa9ecbf
  pending_signals_list.go:231   0xa9ecb0                488b33                  MOVQ 0(BX), SI
  pending_signals_list.go:231   0xa9ecb3                e808a59eff              CALL runtime.gcWriteBarrier2(SB)
  pending_signals_list.go:231   0xa9ecb8                498903                  MOVQ AX, 0(R11)
  pending_signals_list.go:231   0xa9ecbb                49897308                MOVQ SI, 0x8(R11)
  pending_signals_list.go:231   0xa9ecbf                488903                  MOVQ AX, 0(BX)
  pending_signals_list.go:115   0xa9ecc2                eb1d                    JMP 0xa9ece1
  pending_signals_list.go:117   0xa9ecc4                833df5297d0100          CMPL runtime.writeBarrier(SB), $0x0
  pending_signals_list.go:117   0xa9eccb                7410                    JE 0xa9ecdd
  pending_signals_list.go:117   0xa9eccd                488b1cca                MOVQ 0(DX)(CX*8), BX
  pending_signals_list.go:117   0xa9ecd1                e8eaa49eff              CALL runtime.gcWriteBarrier2(SB)
  pending_signals_list.go:117   0xa9ecd6                498903                  MOVQ AX, 0(R11)
  pending_signals_list.go:117   0xa9ecd9                49895b08                MOVQ BX, 0x8(R11)
  pending_signals_list.go:117   0xa9ecdd                488904ca                MOVQ AX, 0(DX)(CX*8)
  pending_signals_list.go:120   0xa9ece1                833dd8297d0100          CMPL runtime.writeBarrier(SB), $0x0
  pending_signals_list.go:120   0xa9ece8                7411                    JE 0xa9ecfb
  pending_signals_list.go:120   0xa9ecea                488b5cca08              MOVQ 0x8(DX)(CX*8), BX
  pending_signals_list.go:120   0xa9ecef                e8cca49eff              CALL runtime.gcWriteBarrier2(SB)
  pending_signals_list.go:120   0xa9ecf4                498903                  MOVQ AX, 0(R11)
  pending_signals_list.go:120   0xa9ecf7                49895b08                MOVQ BX, 0x8(R11)
  pending_signals_list.go:120   0xa9ecfb                488944ca08              MOVQ AX, 0x8(DX)(CX*8)

After this change:

TEXT gvisor.dev/gvisor/pkg/sentry/kernel.(*pendingSignals).enqueue(SB) pkg/sentry/kernel/pending_signals.go
...
  pending_signals.go:101        0xa9c186                488d1d937fbd00          LEAQ gvisor.dev/gvisor/pkg/ilist..dict.List[*gvisor.dev/gvisor/pkg/sentry/kernel.pendingSignal](SB), BX
  pending_signals.go:101        0xa9c18d                e82e1d0500              CALL gvisor.dev/gvisor/pkg/ilist.(*List[go.shape.*gvisor.dev/gvisor/pkg/sentry/kernel.pendingSignal]).PushBack(SB)
...

TEXT gvisor.dev/gvisor/pkg/ilist.(*List[go.shape.*gvisor.dev/gvisor/pkg/sentry/kernel.pendingSignal]).PushBack(SB) pkg/ilist/list.go
  list.go:124           0xaedec0                55                      PUSHQ BP
  list.go:124           0xaedec1                4889e5                  MOVQ SP, BP
  list.go:124           0xaedec4                4883ec18                SUBQ $0x18, SP
  list.go:128           0xaedec8                4889442428              MOVQ AX, 0x28(SP)
  list.go:128           0xaedecd                48895c2430              MOVQ BX, 0x30(SP)
  list.go:128           0xaeded2                48894c2438              MOVQ CX, 0x38(SP)
  list.go:126           0xaeded7                488d5308                LEAQ 0x8(BX), DX
  list.go:126           0xaededb                4889542410              MOVQ DX, 0x10(SP)
  list.go:126           0xaedee0                488b7308                MOVQ 0x8(BX), SI
  list.go:126           0xaedee4                4889c8                  MOVQ CX, AX
  list.go:126           0xaedee7                31db                    XORL BX, BX
  list.go:126           0xaedee9                ffd6                    CALL SI
  list.go:127           0xaedeeb                488b542430              MOVQ 0x30(SP), DX
  list.go:127           0xaedef0                488d4a10                LEAQ 0x10(DX), CX
  list.go:127           0xaedef4                488b7210                MOVQ 0x10(DX), SI
  list.go:127           0xaedef8                488b7c2428              MOVQ 0x28(SP), DI
  list.go:127           0xaedefd                488b5f08                MOVQ 0x8(DI), BX
  list.go:127           0xaedf01                488b442438              MOVQ 0x38(SP), AX
  list.go:127           0xaedf06                4889ca                  MOVQ CX, DX
  list.go:127           0xaedf09                ffd6                    CALL SI
  list.go:128           0xaedf0b                488b4c2428              MOVQ 0x28(SP), CX
  list.go:128           0xaedf10                488b4108                MOVQ 0x8(CX), AX
  list.go:128           0xaedf14                4885c0                  TESTQ AX, AX
  list.go:128           0xaedf17                7529                    JNE 0xaedf42
  list.go:131           0xaedf19                833da037780100          CMPL runtime.writeBarrier(SB), $0x0
  list.go:131           0xaedf20                7507                    JNE 0xaedf29
  list.go:131           0xaedf22                488b542438              MOVQ 0x38(SP), DX
  list.go:131           0xaedf27                eb14                    JMP 0xaedf3d
  list.go:131           0xaedf29                488b01                  MOVQ 0(CX), AX
  list.go:131           0xaedf2c                e88fb299ff              CALL runtime.gcWriteBarrier2(SB)
  list.go:131           0xaedf31                488b542438              MOVQ 0x38(SP), DX
  list.go:131           0xaedf36                498913                  MOVQ DX, 0(R11)
  list.go:131           0xaedf39                49894308                MOVQ AX, 0x8(R11)
  list.go:131           0xaedf3d                488911                  MOVQ DX, 0(CX)
  list.go:131           0xaedf40                eb1f                    JMP 0xaedf61
  list.go:129           0xaedf42                488b4c2430              MOVQ 0x30(SP), CX
  list.go:129           0xaedf47                488b4908                MOVQ 0x8(CX), CX
  list.go:129           0xaedf4b                488b5c2438              MOVQ 0x38(SP), BX
  list.go:129           0xaedf50                488b542410              MOVQ 0x10(SP), DX
  list.go:129           0xaedf55                ffd1                    CALL CX
  list.go:134           0xaedf57                488b4c2428              MOVQ 0x28(SP), CX
  list.go:134           0xaedf5c                488b542438              MOVQ 0x38(SP), DX
  list.go:134           0xaedf61                833d5837780100          CMPL runtime.writeBarrier(SB), $0x0
  list.go:134           0xaedf68                7410                    JE 0xaedf7a
  list.go:134           0xaedf6a                488b4108                MOVQ 0x8(CX), AX
  list.go:134           0xaedf6e                e84db299ff              CALL runtime.gcWriteBarrier2(SB)
  list.go:134           0xaedf73                498913                  MOVQ DX, 0(R11)
  list.go:134           0xaedf76                49894308                MOVQ AX, 0x8(R11)
  list.go:134           0xaedf7a                48895108                MOVQ DX, 0x8(CX)
  list.go:135           0xaedf7e                4883c418                ADDQ $0x18, SP
  list.go:135           0xaedf82                5d                      POPQ BP
  list.go:135           0xaedf83                c3                      RET

Consequently, I don't think we should merge this change.

tamird · 2026-01-09T18:29:59Z

Thanks very much for the analysis! That makes sense, at least for ilist. How about the second commit that touches refs?

nixprime · 2026-01-09T18:51:47Z

I think the refs case is fine, with some changes:

The LoggingPolicy type argument has the same same-shape problem, resulting in an indirect call to LoggingPolicy.enabled() in every IncRef/TryIncRef/DecRef:

TEXT gvisor.dev/gvisor/pkg/refs.(*Refs[go.shape.4a0f57fe00797ca298fdef4b77d40015cc906bef1de26f24611ac34cec347463,go.shape.struct {}]).IncRef(SB) pkg/refs/refs_template.go
  refs_template.go:97   0x9da240                55                      PUSHQ BP
  refs_template.go:97   0x9da241                4889e5                  MOVQ SP, BP
  refs_template.go:97   0x9da244                4883ec60                SUBQ $0x60, SP
  refs_template.go:99   0x9da248                4889442470              MOVQ AX, 0x70(SP)
  refs_template.go:99   0x9da24d                48895c2478              MOVQ BX, 0x78(SP)
  refs_template.go:98   0x9da252                90                      NOPL
  aligned_64bit.go:88   0x9da253                b901000000              MOVL $0x1, CX
  aligned_64bit.go:88   0x9da258                f0480fc108              LOCK XADDQ CX, 0(AX)
  aligned_64bit.go:88   0x9da25d                48894c2428              MOVQ CX, 0x28(SP)
  refs_template.go:99   0x9da262                488b5310                MOVQ 0x10(BX), DX
  refs_template.go:85   0x9da266                488b02                  MOVQ 0(DX), AX
  refs_template.go:85   0x9da269                ffd0                    CALL AX
  aligned_64bit.go:88   0x9da26b                488b4c2428              MOVQ 0x28(SP), CX
  aligned_64bit.go:88   0x9da270                48ffc1                  INCQ CX
  refs_template.go:85   0x9da273                84c0                    TESTL AL, AL
  refs_template.go:99   0x9da275                7420                    JE 0x9da297
  aligned_64bit.go:88   0x9da277                48894c2428              MOVQ CX, 0x28(SP)
  refs_template.go:100  0x9da27c                488b542478              MOVQ 0x78(SP), DX
  refs_template.go:100  0x9da281                488b82a0000000          MOVQ 0xa0(DX), AX
  refs_template.go:100  0x9da288                488b5c2470              MOVQ 0x70(SP), BX
  refs_template.go:100  0x9da28d                e8ae47beff              CALL gvisor.dev/gvisor/pkg/refs.LogIncRef(SB)
...

I think you can avoid this by making the logging policies distinct shapes instead, as in https://github.com/google/gvisor/blob/master/pkg/bpf/input_bytes.go.

An ergonomic complaint: having to explicitly specify refs.LoggingDisabled everywhere is a bit obnoxious, since the intended use of the feature is that it's disabled by default, and selectively enabled to debug refcounting problems for specific types. Is it possible to define e.g. type RefsBase[T any, L LoggingPolicy] struct{...}; type Refs[T any] = RefsBase[T, LoggingDisabled]; type LoggedRefs[T any] = RefsBase[T, LoggingEnabled]?

tamird · 2026-01-10T20:00:53Z

Done. Required some more go_stateify surgery, please have a look!

avagin · 2026-01-11T00:14:10Z

Is there any benefits of using Go generics?

We implement limited generics support: generic types must be monorphized by wrapping them with a struct that embeds them and is annotated with the new annotation `// +stateify transparent` which will generate a SaverLoader implementation that delegates to the wrapped type. This sidesteps the problem of generating a unique StateTypeName implementation for the generic type itself while keeping most of the code generated.

tamird · 2026-01-11T12:54:10Z

Is there any benefits of using Go generics?

There's a big ergonomics benefit. Using bazel in editors is not very well supported, so anyone working on gvisor without the benefit of internal tooling can't easily jump to definition on types generated by go_generics.

tamird · 2026-01-12T18:30:13Z

Could I please get a CI run on this?

nixprime · 2026-01-12T22:02:13Z

The loggingPolicy thing in refs didn't work, there's still a runtime test and branching:

TEXT gvisor.dev/gvisor/pkg/refs.(*RefsBase[go.shape.struct { gvisor.dev/gvisor/pkg/buffer.chunkRefs; gvisor.dev/gvisor/pkg/buffer.data []uint8 },go.shape.uint8]).IncRef(SB) pkg/refs/refs_template.go
  ...
  aligned_64bit.go:88   0x7aa94e                f0480fc108              LOCK XADDQ CX, 0(AX)
  refs_template.go:113  0x7aa953                488b5308                MOVQ 0x8(BX), DX
  refs_template.go:92   0x7aa957                488b5260                MOVQ 0x60(DX), DX
  refs_template.go:92   0x7aa95b                0f1f440000              NOPL 0(AX)(AX*1)
  refs_template.go:92   0x7aa960                4885d2                  TESTQ DX, DX
  refs_template.go:92   0x7aa963                0f844e010000            JE 0x7aaab7
  refs_template.go:92   0x7aa969                4889442470              MOVQ AX, 0x70(SP)
  refs_template.go:92   0x7aa96e                48895c2478              MOVQ BX, 0x78(SP)
  refs_template.go:92   0x7aa973                448b4210                MOVL 0x10(DX), R8
  aligned_64bit.go:88   0x7aa977                48ffc1                  INCQ CX
  aligned_64bit.go:88   0x7aa97a                660f1f440000            NOPW 0(AX)(AX*1)
  refs_template.go:92   0x7aa980                4181f8a6ef4c1f          CMPL R8, $0x1f4cefa6
  refs_template.go:92   0x7aa987                7539                    JNE 0x7aa9c2
  refs_template.go:95   0x7aa989                488d35b0e1a100          LEAQ 0xa1e1b0(IP), SI
  refs_template.go:95   0x7aa990                4839f2                  CMPQ DX, SI
  refs_template.go:95   0x7aa993                0f851e010000            JNE 0x7aaab7
  aligned_64bit.go:88   0x7aa999                48894c2428              MOVQ CX, 0x28(SP)
  refs_template.go:114  0x7aa99e                488b93d8000000          MOVQ 0xd8(BX), DX
  refs_template.go:114  0x7aa9a5                4889c3                  MOVQ AX, BX
  refs_template.go:114  0x7aa9a8                4889d0                  MOVQ DX, AX
  refs_template.go:114  0x7aa9ab                e8f03de1ff              CALL gvisor.dev/gvisor/pkg/refs.LogIncRef(SB)

It looks like it doesn't work in bpf/input_bytes.go anymore either... From experimentation, it looks like it works if bpf.load*() is inlined into a call site where endian is specified explicitly, but when it's inlined into e.g. bpf.Exec[endian Endianness]() the type-switch occurs at runtime. I'm asking the Go team if there's anything we can do about this.

nixprime · 2026-01-12T22:20:58Z

Apparently this is golang/go#59591.

I tried inlining LogRefs() into IncRef() etc. (var l L; if _, ok := any(l).(loggingEnabled); ok { ... }), but this didn't help the generated code.

tamird force-pushed the refs-generics branch from f1c203e to b0e7be0 Compare January 9, 2026 14:53

avagin requested review from avagin, ayushr2 and nixprime January 9, 2026 15:50

tamird force-pushed the refs-generics branch from b0e7be0 to 9b6d47c Compare January 9, 2026 18:15

nixprime reviewed Jan 9, 2026

View reviewed changes

go_stateify: extract local

a8c49e5

tamird force-pushed the refs-generics branch from 9b6d47c to cfe38f7 Compare January 10, 2026 20:00

tamird changed the title ~~ilist,refs: replace go_generics with Go generics~~ refs: replace go_generics with Go generics Jan 10, 2026

tamird added 2 commits January 11, 2026 07:52

refs: replace go_generics with Go generics

d47f0f4

tamird force-pushed the refs-generics branch from cfe38f7 to d47f0f4 Compare January 11, 2026 21:49

avagin added the ready to pull label Jan 12, 2026

tamird mentioned this pull request Jan 13, 2026

atomicptr: use atomic.Pointer #12483

Open

refs: replace go_generics with Go generics #12470

Are you sure you want to change the base?

refs: replace go_generics with Go generics #12470

Conversation

tamird commented Jan 9, 2026

Uh oh!

tamird commented Jan 9, 2026

Uh oh!

avagin commented Jan 9, 2026

Uh oh!

tamird commented Jan 9, 2026

Uh oh!

ayushr2 commented Jan 9, 2026

Uh oh!

nixprime left a comment

Choose a reason for hiding this comment

Uh oh!

tamird commented Jan 9, 2026

Uh oh!

nixprime commented Jan 9, 2026

Uh oh!

tamird commented Jan 10, 2026

Uh oh!

avagin commented Jan 11, 2026

Uh oh!

tamird commented Jan 11, 2026

Uh oh!

tamird commented Jan 12, 2026

Uh oh!

nixprime commented Jan 12, 2026

Uh oh!

nixprime commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants