Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: Enable PyTorch2 Batching Tests#8814

Open
mattwittwer wants to merge 8 commits into
mainfrom
mwittwer/enable_pytorch2_batching
Open

feat: Enable PyTorch2 Batching Tests#8814
mattwittwer wants to merge 8 commits into
mainfrom
mwittwer/enable_pytorch2_batching

Conversation

@mattwittwer
Copy link
Copy Markdown
Contributor

@mattwittwer mattwittwer commented Jun 2, 2026

What does the PR do?

Makes the AOTI test models batch‑capable and adds coverage: the simple add/sub model is exported with a dynamic batch dim and max_batch_size: 8; a new sequence (implicit‑state accumulator) model + config is added to gen_qa_implicit_models.py and wired into gen_qa_model_repository; and torch_aoti_infer_test.py gains batched inference cases (batch 1/4/8 across dtypes) plus a sequence test class (single + interleaved sequences), with test.sh updated to pull and run them.

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

triton-inference-server/pytorch_backend#196

Where should the reviewer start?

Test plan:

  • CI Pipeline ID:

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@mattwittwer mattwittwer self-assigned this Jun 2, 2026
@mattwittwer mattwittwer changed the title draft: enable pytorch2 batching feat: enable pytorch2 batching Jun 4, 2026
@mattwittwer mattwittwer changed the title feat: enable pytorch2 batching feat: Enable PyTorch2 Batching Jun 4, 2026
@mattwittwer mattwittwer changed the title feat: Enable PyTorch2 Batching feat: Enable PyTorch2 Batching Tests Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant