Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 0c17462

Browse files
morealmoreal
authored
Introduce /upgrade-pylib-next (#6827)
* Simplify upgrade-pylib.md by removing manual copy steps handled by quick command * Let upgrade-pylib run auto-mark instead marking manually * Let upgrade-pylib a separate commit from updating module * Correct commands' permissions * Add open PR check to upgrade-pylib-next command * Fix dependency pattern example in upgrade-pylib-next command * Let upgrade-pylib command to invetigate failing tests * Add pre-commit review to invetigate-test-failure command * Use deps command to get dependent tests in upgrade-pylib workflow
1 parent f6d562e commit 0c17462

File tree

3 files changed

+142
-32
lines changed

3 files changed

+142
-32
lines changed

‎.claude/commands/investigate-test-failure.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
allowed-tools: Bash(python3:*), Bash(cargo run:*), Read, Grep, Glob, Bash(git add:*), Bash(git commit:*), Bash(cargo fmt:*), Bash(git diff:*), Task
3+
---
4+
5+
# Investigate Test Failure
6+
7+
Investigate why a specific test is failing and determine if it can be fixed or needs an issue.
8+
9+
## Arguments
10+
- `$ARGUMENTS`: Failed test identifier (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`)
11+
12+
## Steps
13+
14+
1. **Analyze failure cause**
15+
- Read the test code
16+
- Analyze failure message/traceback
17+
- Check related RustPython code
18+
19+
2. **Verify behavior in CPython**
20+
- Run the test with `python3 -m unittest` to confirm expected behavior
21+
- Document the expected output
22+
23+
3. **Determine fix feasibility**
24+
- **Simple fix** (import issues, small logic bugs): Fix code → Run `cargo fmt --all` → Pre-commit review → Commit
25+
- **Complex fix** (major unimplemented features): Collect issue info and report to user
26+
27+
**Pre-commit review process**:
28+
- Run `git diff` to see the changes
29+
- Use Task tool with `general-purpose` subagent to review:
30+
- Compare implementation against cpython/ source code
31+
- Verify the fix aligns with CPython behavior
32+
- Check for any missed edge cases
33+
- Proceed to commit only after review passes
34+
35+
4. **For complex issues - Collect issue information**
36+
Following `.github/ISSUE_TEMPLATE/report-incompatibility.md` format:
37+
38+
- **Feature**: Description of missing/broken Python feature
39+
- **Minimal reproduction code**: Smallest code that reproduces the issue
40+
- **CPython behavior**: Result when running with python3
41+
- **RustPython behavior**: Result when running with cargo run
42+
- **Python Documentation link**: Link to relevant CPython docs
43+
44+
Report collected information to the user. Issue creation is done only upon user request.
45+
46+
Example issue creation command:
47+
```
48+
gh issue create --template report-incompatibility.md --title "..." --body "..."
49+
```

‎.claude/commands/upgrade-pylib-next.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
allowed-tools: Skill(upgrade-pylib), Bash(gh pr list:*)
3+
---
4+
5+
# Upgrade Next Python Library
6+
7+
Find the next Python library module ready for upgrade and run `/upgrade-pylib` for it.
8+
9+
## Current TODO Status
10+
11+
!`cargo run --release -- scripts/update_lib todo 2>/dev/null`
12+
13+
## Open Upgrade PRs
14+
15+
!`gh pr list --search "Update in:title" --json number,title --template '{{range .}}#{{.number}} {{.title}}{{"\n"}}{{end}}'`
16+
17+
## Instructions
18+
19+
From the TODO list above, find modules matching these patterns (in priority order):
20+
21+
1. `[ ] [no deps]` - Modules with no dependencies (can be upgraded immediately)
22+
2. `[ ] [0/n]` - Modules where all dependencies are already upgraded (e.g., `[0/3]`, `[0/5]`)
23+
24+
These patterns indicate modules that are ready to upgrade without blocking dependencies.
25+
26+
**Important**: Skip any modules that already have an open PR in the "Open Upgrade PRs" list above.
27+
28+
**After identifying a suitable module**, run:
29+
```
30+
/upgrade-pylib <module_name>
31+
```
32+
33+
If no modules match these criteria, inform the user that all eligible modules have dependencies that need to be upgraded first.

‎.claude/commands/upgrade-pylib.md‎

Lines changed: 60 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*)
3+
---
4+
15
# Upgrade Python Library from CPython
26

37
Upgrade a Python standard library module from CPython to RustPython.
@@ -23,24 +27,19 @@ This helps improve the tooling for future upgrades.
2327

2428
## Steps
2529

26-
1. **Delete existing library in Lib/**
27-
- If `Lib/$ARGUMENTS.py` exists, delete it
28-
- If `Lib/$ARGUMENTS/` directory exists, delete it
29-
30-
2. **Copy from cpython/Lib/**
31-
- If `cpython/Lib/$ARGUMENTS.py` exists, copy it to `Lib/$ARGUMENTS.py`
32-
- If `cpython/Lib/$ARGUMENTS/` directory exists, copy it to `Lib/$ARGUMENTS/`
33-
34-
3. **Upgrade tests (quick upgrade with update_lib)**
35-
- Run: `python3 scripts/update_lib quick cpython/Lib/test/test_$ARGUMENTS.py` (single file)
36-
- Or: `python3 scripts/update_lib quick cpython/Lib/test/test_$ARGUMENTS/` (directory)
30+
1. **Run quick upgrade with update_lib**
31+
- Run: `python3 scripts/update_lib quick $ARGUMENTS` (module name)
32+
- Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS.py` (library file path)
33+
- Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS/` (library directory path)
3734
- This will:
35+
- Copy library files (delete existing `Lib/$ARGUMENTS.py` or `Lib/$ARGUMENTS/`, then copy from `cpython/Lib/`)
3836
- Patch test files preserving existing RustPython markers
3937
- Run tests and auto-mark new test failures (not regressions)
4038
- Remove `@unittest.expectedFailure` from tests that now pass
41-
- **Handle warnings**: If you see warnings like `WARNING: TestCFoo does not exist in remote file`, it means the class structure changed and markers couldn't be transferred automatically. These need to be manually restored in step 4 or added in step 5.
39+
- Create a git commit with the changes
40+
- **Handle warnings**: If you see warnings like `WARNING: TestCFoo does not exist in remote file`, it means the class structure changed and markers couldn't be transferred automatically. These need to be manually restored in step 2 or added in step 3.
4241

43-
4. **Review git diff and restore RUSTPYTHON-specific changes**
42+
2. **Review git diff and restore RUSTPYTHON-specific changes**
4443
- Run `git diff Lib/test/test_$ARGUMENTS` to review all changes
4544
- **Only restore changes that have explicit `RUSTPYTHON` comments**. Look for:
4645
- `# XXX: RUSTPYTHON` or `# XXX RUSTPYTHON` - Comments marking RustPython-specific code modifications
@@ -49,13 +48,46 @@ This helps improve the tooling for future upgrades.
4948
- **Do NOT restore other diff changes** - these are likely upstream CPython changes, not RustPython-specific modifications
5049
- When restoring, preserve the original context and formatting
5150

52-
5. **Verify tests**
53-
- Run: `cargo run --release -- -m test test_$ARGUMENTS -v`
54-
- The `-v` flag shows detailed output to identify which tests fail and why
55-
- For each new failure, add appropriate markers based on the failure type:
56-
- **Test assertion failure** → `@unittest.expectedFailure` with `# TODO: RUSTPYTHON` comment
57-
- **Panic/crash** → `@unittest.skip("TODO: RUSTPYTHON; <panic message>")`
58-
- **Class-specific markers**: If a test fails only in the C implementation (TestCFoo) but passes in the Python implementation (TestPyFoo), or vice versa, add the marker to the specific subclass, not the base class:
51+
3. **Investigate test failures with subagent**
52+
- First, get dependent tests using the deps command:
53+
```
54+
cargo run --release -- scripts/update_lib deps $ARGUMENTS
55+
```
56+
- Look for the line `- [ ] $ARGUMENTS: test_xxx test_yyy ...` to get the direct dependent tests
57+
- Run those tests to collect failures:
58+
```
59+
cargo run --release -- -m test test_xxx test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):"
60+
```
61+
- For example, if deps output shows `- [ ] linecache: test_bdb test_inspect test_linecache test_traceback test_zipimport`, run:
62+
```
63+
cargo run --release -- -m test test_bdb test_inspect test_linecache test_traceback test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):"
64+
```
65+
- For each failure, use the Task tool with `general-purpose` subagent to investigate:
66+
- Subagent should follow the `/investigate-test-failure` skill workflow
67+
- Pass the failed test identifier as the argument (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`)
68+
- If subagent can fix the issue easily: fix and commit
69+
- If complex issue: subagent collects issue info and reports back (issue creation on user request only)
70+
- Using subagent prevents context pollution in the main conversation
71+
72+
4. **Mark remaining test failures with auto-mark**
73+
- Run: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS.py --mark-failure`
74+
- Or for directory: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS/ --mark-failure`
75+
- This will:
76+
- Run tests and mark ALL failing tests with `@unittest.expectedFailure`
77+
- Remove `@unittest.expectedFailure` from tests that now pass
78+
- **Note**: The `--mark-failure` flag marks all failures including regressions. Review the changes before committing.
79+
80+
5. **Handle panics manually**
81+
- If any tests cause panics/crashes (not just assertion failures), they need `@unittest.skip` instead:
82+
```python
83+
@unittest.skip("TODO: RUSTPYTHON; panics with 'index out of bounds'")
84+
def test_crashes(self):
85+
...
86+
```
87+
- auto-mark cannot detect panics automatically - check the test output for crash messages
88+
89+
6. **Handle class-specific failures**
90+
- If a test fails only in the C implementation (TestCFoo) but passes in the Python implementation (TestPyFoo), or vice versa, move the marker to the specific subclass:
5991
```python
6092
# Base class - no marker here
6193
class TestFoo:
@@ -70,25 +102,21 @@ This helps improve the tooling for future upgrades.
70102
def test_something(self):
71103
return super().test_something()
72104
```
73-
- **New tests from CPython**: The upgrade may bring in entirely new tests that didn't exist before. These won't have any RUSTPYTHON markers in the diff - they just need to be tested and marked if they fail.
74-
- Example markers:
75-
```python
76-
# TODO: RUSTPYTHON
77-
@unittest.expectedFailure
78-
def test_something(self):
79-
...
80105
81-
# TODO: RUSTPYTHON
82-
@unittest.skip("TODO: RUSTPYTHON; panics with 'index out of bounds'")
83-
def test_crashes(self):
84-
...
85-
```
106+
7. **Commit the test fixes**
107+
- Run: `git add -u && git commit -m "Mark failing tests"`
108+
- This creates a separate commit for the test markers added in steps 2-6
86109
87110
## Example Usage
88111
```
112+
# Using module names (recommended)
89113
/upgrade-pylib inspect
90114
/upgrade-pylib json
91115
/upgrade-pylib asyncio
116+
117+
# Using library paths (alternative)
118+
/upgrade-pylib cpython/Lib/inspect.py
119+
/upgrade-pylib cpython/Lib/json/
92120
```
93121
94122
## Example: Restoring RUSTPYTHON changes

0 commit comments

Comments
 (0)