Thanks to visit codestin.com
Credit goes to GitHub.com

Skip to content

Introduce /upgrade-pylib-next #6827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 9 commits into from
Jan 30, 2026
Merged

Introduce /upgrade-pylib-next #6827

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6b3e342
Simplify upgrade-pylib.md by removing manual copy steps handled by qu…
moreal Jan 21, 2026
70e7a6d
Let upgrade-pylib run auto-mark instead marking manually
moreal Jan 21, 2026
37bdfd6
Let upgrade-pylib a separate commit from updating module
moreal Jan 21, 2026
35ceed8
Correct commands' permissions
moreal Jan 21, 2026
79babbc
Add open PR check to upgrade-pylib-next command
moreal Jan 21, 2026
cd1389b
Fix dependency pattern example in upgrade-pylib-next command
moreal Jan 21, 2026
b2c93ca
Let upgrade-pylib command to invetigate failing tests
moreal Jan 21, 2026
2712f2b
Add pre-commit review to invetigate-test-failure command
moreal Jan 23, 2026
01c614d
Use deps command to get dependent tests in upgrade-pylib workflow
moreal Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .claude/commands/investigate-test-failure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
allowed-tools: Bash(python3:*), Bash(cargo run:*), Read, Grep, Glob, Bash(git add:*), Bash(git commit:*), Bash(cargo fmt:*), Bash(git diff:*), Task
---

# Investigate Test Failure

Investigate why a specific test is failing and determine if it can be fixed or needs an issue.

## Arguments
- `$ARGUMENTS`: Failed test identifier (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`)

## Steps

1. **Analyze failure cause**
- Read the test code
- Analyze failure message/traceback
- Check related RustPython code

2. **Verify behavior in CPython**
- Run the test with `python3 -m unittest` to confirm expected behavior
- Document the expected output

3. **Determine fix feasibility**
- **Simple fix** (import issues, small logic bugs): Fix code → Run `cargo fmt --all` → Pre-commit review → Commit
- **Complex fix** (major unimplemented features): Collect issue info and report to user

**Pre-commit review process**:
- Run `git diff` to see the changes
- Use Task tool with `general-purpose` subagent to review:
- Compare implementation against cpython/ source code
- Verify the fix aligns with CPython behavior
- Check for any missed edge cases
- Proceed to commit only after review passes

4. **For complex issues - Collect issue information**
Following `.github/ISSUE_TEMPLATE/report-incompatibility.md` format:

- **Feature**: Description of missing/broken Python feature
- **Minimal reproduction code**: Smallest code that reproduces the issue
- **CPython behavior**: Result when running with python3
- **RustPython behavior**: Result when running with cargo run
- **Python Documentation link**: Link to relevant CPython docs

Report collected information to the user. Issue creation is done only upon user request.

Example issue creation command:
```
gh issue create --template report-incompatibility.md --title "..." --body "..."
```
33 changes: 33 additions & 0 deletions .claude/commands/upgrade-pylib-next.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
allowed-tools: Skill(upgrade-pylib), Bash(gh pr list:*)
---

# Upgrade Next Python Library

Find the next Python library module ready for upgrade and run `/upgrade-pylib` for it.

## Current TODO Status

!`cargo run --release -- scripts/update_lib todo 2>/dev/null`

## Open Upgrade PRs

!`gh pr list --search "Update in:title" --json number,title --template '{{range .}}#{{.number}} {{.title}}{{"\n"}}{{end}}'`

## Instructions

From the TODO list above, find modules matching these patterns (in priority order):

1. `[ ] [no deps]` - Modules with no dependencies (can be upgraded immediately)
2. `[ ] [0/n]` - Modules where all dependencies are already upgraded (e.g., `[0/3]`, `[0/5]`)

These patterns indicate modules that are ready to upgrade without blocking dependencies.

**Important**: Skip any modules that already have an open PR in the "Open Upgrade PRs" list above.

**After identifying a suitable module**, run:
```
/upgrade-pylib <module_name>
```

If no modules match these criteria, inform the user that all eligible modules have dependencies that need to be upgraded first.
92 changes: 60 additions & 32 deletions .claude/commands/upgrade-pylib.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*)
---
Comment on lines +1 to +3
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Add missing deps subcommand to allowed-tools.

The workflow in step 3 (line 54) uses cargo run --release -- scripts/update_lib deps, but the allowed-tools header only permits quick and auto-mark subcommands. This will cause the command to be blocked during execution.

🔧 Proposed fix 
 ---
-allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*)
+allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*), Bash(cargo run:*)
 ---

Note: This allows cargo run for both the deps command (line 54) and the test execution commands (lines 59, 63).

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
---
allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*)
---
---
allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*), Bash(cargo run:*)
---
🤖 Prompt for AI Agents 
In @.claude/commands/upgrade-pylib.md around lines 1 - 3, The allowed-tools
header currently permits only Bash(git add:*), Bash(git commit:*), Bash(python3
scripts/update_lib quick:*), and Bash(python3 scripts/update_lib auto-mark:*),
but the workflow runs cargo run --release -- scripts/update_lib deps; update the
allowed-tools header to also include Bash(python3 scripts/update_lib deps:*) (or
the equivalent pattern used for quick/auto-mark) so the `deps` subcommand is
allowed; ensure the new entry matches the existing pattern for `quick` and
`auto-mark` to permit the deps invocation and related test commands.


# Upgrade Python Library from CPython

Upgrade a Python standard library module from CPython to RustPython.
Expand All @@ -23,24 +27,19 @@ This helps improve the tooling for future upgrades.

## Steps

1. **Delete existing library in Lib/**
- If `Lib/$ARGUMENTS.py` exists, delete it
- If `Lib/$ARGUMENTS/` directory exists, delete it

2. **Copy from cpython/Lib/**
- If `cpython/Lib/$ARGUMENTS.py` exists, copy it to `Lib/$ARGUMENTS.py`
- If `cpython/Lib/$ARGUMENTS/` directory exists, copy it to `Lib/$ARGUMENTS/`

3. **Upgrade tests (quick upgrade with update_lib)**
- Run: `python3 scripts/update_lib quick cpython/Lib/test/test_$ARGUMENTS.py` (single file)
- Or: `python3 scripts/update_lib quick cpython/Lib/test/test_$ARGUMENTS/` (directory)
1. **Run quick upgrade with update_lib**
- Run: `python3 scripts/update_lib quick $ARGUMENTS` (module name)
- Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS.py` (library file path)
- Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS/` (library directory path)
- This will:
- Copy library files (delete existing `Lib/$ARGUMENTS.py` or `Lib/$ARGUMENTS/`, then copy from `cpython/Lib/`)
- Patch test files preserving existing RustPython markers
- Run tests and auto-mark new test failures (not regressions)
- Remove `@unittest.expectedFailure` from tests that now pass
- **Handle warnings**: If you see warnings like `WARNING: TestCFoo does not exist in remote file`, it means the class structure changed and markers couldn't be transferred automatically. These need to be manually restored in step 4 or added in step 5.
- Create a git commit with the changes
- **Handle warnings**: If you see warnings like `WARNING: TestCFoo does not exist in remote file`, it means the class structure changed and markers couldn't be transferred automatically. These need to be manually restored in step 2 or added in step 3.

4. **Review git diff and restore RUSTPYTHON-specific changes**
2. **Review git diff and restore RUSTPYTHON-specific changes**
- Run `git diff Lib/test/test_$ARGUMENTS` to review all changes
- **Only restore changes that have explicit `RUSTPYTHON` comments**. Look for:
- `# XXX: RUSTPYTHON` or `# XXX RUSTPYTHON` - Comments marking RustPython-specific code modifications
Expand All @@ -49,13 +48,46 @@ This helps improve the tooling for future upgrades.
- **Do NOT restore other diff changes** - these are likely upstream CPython changes, not RustPython-specific modifications
- When restoring, preserve the original context and formatting

5. **Verify tests**
- Run: `cargo run --release -- -m test test_$ARGUMENTS -v`
- The `-v` flag shows detailed output to identify which tests fail and why
- For each new failure, add appropriate markers based on the failure type:
- **Test assertion failure** → `@unittest.expectedFailure` with `# TODO: RUSTPYTHON` comment
- **Panic/crash** → `@unittest.skip("TODO: RUSTPYTHON; <panic message>")`
- **Class-specific markers**: If a test fails only in the C implementation (TestCFoo) but passes in the Python implementation (TestPyFoo), or vice versa, add the marker to the specific subclass, not the base class:
3. **Investigate test failures with subagent**
- First, get dependent tests using the deps command:
```
cargo run --release -- scripts/update_lib deps $ARGUMENTS
```
- Look for the line `- [ ] $ARGUMENTS: test_xxx test_yyy ...` to get the direct dependent tests
- Run those tests to collect failures:
```
cargo run --release -- -m test test_xxx test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):"
```
- For example, if deps output shows `- [ ] linecache: test_bdb test_inspect test_linecache test_traceback test_zipimport`, run:
```
cargo run --release -- -m test test_bdb test_inspect test_linecache test_traceback test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):"
```
- For each failure, use the Task tool with `general-purpose` subagent to investigate:
- Subagent should follow the `/investigate-test-failure` skill workflow
- Pass the failed test identifier as the argument (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`)
- If subagent can fix the issue easily: fix and commit
- If complex issue: subagent collects issue info and reports back (issue creation on user request only)
- Using subagent prevents context pollution in the main conversation

Comment on lines +51 to +71
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add language identifiers to code blocks in step 3.

Multiple code blocks in the test failure investigation step are missing language specifications, which reduces readability and prevents syntax highlighting.

📝 Proposed fix 
 3. **Investigate test failures with subagent**
    - First, get dependent tests using the deps command:
-     ```
+     ```bash
      cargo run --release -- scripts/update_lib deps $ARGUMENTS
      ```
    - Look for the line `- [ ] $ARGUMENTS: test_xxx test_yyy ...` to get the direct dependent tests
    - Run those tests to collect failures:
-     ```
+     ```bash
      cargo run --release -- -m test test_xxx test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):"
      ```
    - For example, if deps output shows `- [ ] linecache: test_bdb test_inspect test_linecache test_traceback test_zipimport`, run:
-     ```
+     ```bash
      cargo run --release -- -m test test_bdb test_inspect test_linecache test_traceback test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):"
      ```
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
3. **Investigate test failures with subagent**
- First, get dependent tests using the deps command:
```
cargo run --release -- scripts/update_lib deps $ARGUMENTS
```
- Look for the line `- [ ] $ARGUMENTS: test_xxx test_yyy ...` to get the direct dependent tests
- Run those tests to collect failures:
```
cargo run --release -- -m test test_xxx test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):"
```
- For example, if deps output shows `- [ ] linecache: test_bdb test_inspect test_linecache test_traceback test_zipimport`, run:
```
cargo run --release -- -m test test_bdb test_inspect test_linecache test_traceback test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):"
```
- For each failure, use the Task tool with `general-purpose` subagent to investigate:
- Subagent should follow the `/investigate-test-failure` skill workflow
- Pass the failed test identifier as the argument (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`)
- If subagent can fix the issue easily: fix and commit
- If complex issue: subagent collects issue info and reports back (issue creation on user request only)
- Using subagent prevents context pollution in the main conversation
3. **Investigate test failures with subagent**
- First, get dependent tests using the deps command:
🧰 Tools
🪛 LanguageTool

[style] ~68-~68: Consider using a different verb for a more formal wording.
Context: ...getsource_reload`) - If subagent can fix the issue easily: fix and commit - I...

(FIX_RESOLVE)

🪛 markdownlint-cli2 (0.20.0)

[warning] 53-53: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

[warning] 58-58: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

[warning] 62-62: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
In @.claude/commands/upgrade-pylib.md around lines 51 - 71, Step 3's markdown
code blocks lack language identifiers; update the code fences around the shell
commands in the “Investigate test failures with subagent” section to use ```bash
(or ```sh) for the three examples containing the commands `cargo run --release
-- scripts/update_lib deps $ARGUMENTS`, `cargo run --release -- -m test test_xxx
test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):"`, and the example `cargo run
--release -- -m test test_bdb test_inspect test_linecache test_traceback
test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):"` so they render with proper
syntax highlighting.

4. **Mark remaining test failures with auto-mark**
- Run: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS.py --mark-failure`
- Or for directory: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS/ --mark-failure`
- This will:
- Run tests and mark ALL failing tests with `@unittest.expectedFailure`
- Remove `@unittest.expectedFailure` from tests that now pass
- **Note**: The `--mark-failure` flag marks all failures including regressions. Review the changes before committing.

5. **Handle panics manually**
- If any tests cause panics/crashes (not just assertion failures), they need `@unittest.skip` instead:
```python
@unittest.skip("TODO: RUSTPYTHON; panics with 'index out of bounds'")
def test_crashes(self):
...
```
- auto-mark cannot detect panics automatically - check the test output for crash messages

6. **Handle class-specific failures**
- If a test fails only in the C implementation (TestCFoo) but passes in the Python implementation (TestPyFoo), or vice versa, move the marker to the specific subclass:
```python
# Base class - no marker here
class TestFoo:
Expand All @@ -70,25 +102,21 @@ This helps improve the tooling for future upgrades.
def test_something(self):
return super().test_something()
```
- **New tests from CPython**: The upgrade may bring in entirely new tests that didn't exist before. These won't have any RUSTPYTHON markers in the diff - they just need to be tested and marked if they fail.
- Example markers:
```python
# TODO: RUSTPYTHON
@unittest.expectedFailure
def test_something(self):
...

# TODO: RUSTPYTHON
@unittest.skip("TODO: RUSTPYTHON; panics with 'index out of bounds'")
def test_crashes(self):
...
```
7. **Commit the test fixes**
- Run: `git add -u && git commit -m "Mark failing tests"`
- This creates a separate commit for the test markers added in steps 2-6

## Example Usage
```
# Using module names (recommended)
/upgrade-pylib inspect
/upgrade-pylib json
/upgrade-pylib asyncio

# Using library paths (alternative)
/upgrade-pylib cpython/Lib/inspect.py
/upgrade-pylib cpython/Lib/json/
```
Comment on lines 110 to 120
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add language identifier to example usage code block.

The example usage code block is missing a language specification.

📝 Proposed fix 
 ## Example Usage
-```
+```bash
 # Using module names (recommended)
 /upgrade-pylib inspect
 /upgrade-pylib json
 /upgrade-pylib asyncio
 
 # Using library paths (alternative)
 /upgrade-pylib cpython/Lib/inspect.py
 /upgrade-pylib cpython/Lib/json/

</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
## Example Usage
🧰 Tools
🪛 markdownlint-cli2 (0.20.0)

[warning] 111-111: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
In @.claude/commands/upgrade-pylib.md around lines 110 - 120, Update the fenced
"Example Usage" code block to include a language identifier so syntax
highlighting is applied; specifically add "bash" after the opening triple
backticks for the block that contains the /upgrade-pylib usage examples (lines
showing "/upgrade-pylib inspect", "/upgrade-pylib json", "/upgrade-pylib
asyncio" and the path examples) so the block begins with ```bash instead of ```;
no other changes to the content are needed.


## Example: Restoring RUSTPYTHON changes
Expand Down
Loading