Thanks to visit codestin.com
Credit goes to github.com

Skip to content

We can now untokenize PEP 701 syntax, test_tokenize needs an update #111031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
sobolevn opened this issue Oct 18, 2023 · 3 comments
Closed

We can now untokenize PEP 701 syntax, test_tokenize needs an update #111031

sobolevn opened this issue Oct 18, 2023 · 3 comments
Assignees
@sobolevn
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@sobolevn
Copy link
Member

sobolevn commented Oct 18, 2023 •
edited by bedevere-app bot
Loading

Bug report

It states:

cpython/Lib/test/test_tokenize.py

Lines 1911 to 1912 in baefbb2

# TODO: Remove this once we can untokenize PEP 701 syntax
testfiles.remove(os.path.join(tempdir, "test_fstring.py"))

But, it works now 🎉
So, I guess we need to test f strings now, since it is an important part of Python's syntax.

Before:

» ./python.exe -m test test_tokenize
Using random seed 465266543
0:00:00 load avg: 2.63 Run 1 test sequentially
0:00:00 load avg: 2.63 [1/1] test_tokenize

== Tests result: SUCCESS ==

1 test OK.

Total duration: 857 ms
Total tests: run=107
Total test files: run=1/1
Result: SUCCESS

After:

» ./python.exe -m test test_tokenize
Using random seed 1341397923
0:00:00 load avg: 2.58 Run 1 test sequentially
0:00:00 load avg: 2.58 [1/1] test_tokenize

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.2 sec
Total tests: run=107
Total test files: run=1/1
Result: SUCCESS

Linked PRs
@sobolevn sobolevn added type-bug An unexpected behavior, bug, or error tests Tests in the Lib/test dir labels Oct 18, 2023
@sobolevn sobolevn self-assigned this Oct 18, 2023
@sobolevn sobolevn added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Oct 18, 2023
sobolevn added a commit to sobolevn/cpython that referenced this issue Oct 18, 2023
@sobolevn
pythongh-111031: Check test_fstring.py in test_tokenize
a82df84
@bedevere-app bedevere-app bot mentioned this issue Oct 18, 2023
Merged
@sunmy2019
Copy link
Member

All files except for test_fstring.py can be untokenized now.

AlexWaygood pushed a commit that referenced this issue Oct 19, 2023
@sobolevn
gh-111031: Check more files in test_tokenize (#111032)
e9b5399
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 19, 2023
@sobolevn @miss-islington
pythongh-111031: Check more files in test_tokenize (pythonGH-111032)
0e04fac 
(cherry picked from commit e9b5399)

Co-authored-by: Nikita Sobolev <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Oct 19, 2023
Merged
AlexWaygood pushed a commit that referenced this issue Oct 19, 2023
@miss-islington @sobolevn
[3.12] gh-111031: Check more files in test_tokenize (GH-111032) (#1…
40db34c 
…11061)

gh-111031: Check more files in `test_tokenize` (GH-111032)
(cherry picked from commit e9b5399)

Co-authored-by: Nikita Sobolev <[email protected]>
@AlexWaygood
Copy link
Member

As discussed in #111032 (review), it seems we can't, in fact, untokenize PEP-701 syntax just yet (but the linked PRs expanded the test files tested by test_tokenize nonetheless, which is great!).

Anything else to do here @sobolevn, or shall we close this out now? :)

@sobolevn
Copy link
Member Author

Yes, let's close it :)

aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@sobolevn @aisk
pythongh-111031: Check more files in test_tokenize (python#111032)
24bffed
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
@sobolevn @Glyphack
pythongh-111031: Check more files in test_tokenize (python#111032)
d8c5cf6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sobolevn sobolevn

Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sobolevn @sunmy2019 @AlexWaygood