Thanks to visit codestin.com
Credit goes to github.com

Skip to content

can not unparse code with ' in format_spec #112364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
15r10nk opened this issue Nov 24, 2023 · 7 comments
Closed

can not unparse code with ' in format_spec #112364

15r10nk opened this issue Nov 24, 2023 · 7 comments
Labels
3.12 only security fixes 3.13 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@15r10nk
Copy link
Contributor

15r10nk commented Nov 24, 2023 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

The following example shows that it is not possible to unparse a f-string with a ' in the format_spec, but such code can be generated when the f-string is double-quoted.

expected behaviour:
unparse should use different quotes if quotes are part of the format_spec.

This is only a problem in 3.12 and worked in 3.11

import ast

code="""f"{something:'}" """
print("original code:",code)

tree=ast.parse(code)
print("original tree:",ast.dump(tree,indent=2))

new_code=ast.unparse(tree)
print("unparsed code:",new_code)

ast.parse(new_code)

output (Python 3.12.0):

original code: f"{something:'}" 
original tree: Module(
  body=[
    Expr(
      value=JoinedStr(
        values=[
          FormattedValue(
            value=Name(id='something', ctx=Load()),
            conversion=-1,
            format_spec=JoinedStr(
              values=[
                Constant(value="'")]))]))],
  type_ignores=[])
unparsed code: f'{something:'}'
Traceback (most recent call last):
  File "/home/frank/projects/pysource-playground/pysource-codegen/codi.py", line 13, in <module>
    ast.parse(new_code)
  File "/home/frank/.pyenv/versions/3.12.0/lib/python3.12/ast.py", line 52, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 1
    f'{something:'}'
                   ^
SyntaxError: unterminated string literal (detected at line 1)

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs
@15r10nk 15r10nk added the type-bug An unexpected behavior, bug, or error label Nov 24, 2023
@AlexWaygood AlexWaygood added stdlib Python modules in the Lib dir 3.12 only security fixes 3.13 bugs and security fixes labels Nov 24, 2023
@Eclips4
Copy link
Member

Eclips4 commented Nov 24, 2023

cc @pablogsal @lysnikolaou

@pablogsal
Copy link
Member

Will take a look this week

@pablogsal
Copy link
Member

I took a look at this issue and I think switching to a different set of quotes is too complex so the best way foward is that unparse scapes the quote. @isidentical @lysnikolaou WDYT?

@terryjreedy terryjreedy added interpreter-core (Objects, Python, Grammar, and Parser dirs) needs backport to 3.12 only security fixes labels Nov 24, 2023
@terryjreedy
Copy link
Member 

import ast

code1="""f"{something:'}" """
code2="""f'{something:\\'}' """
print(code1)
print(code2)

tree1=ast.parse(code1)
tree2=ast.parse(code2)
print("Trees equal: ", tree1 == tree2)

dump1 = ast.dump(tree1, indent=2)
dump2 = ast.dump(tree2, indent=2)
print("Dumps equal: ", dump1 == dump2)
print("Unparses equal: ", ast.unparse(tree1) == ast.unparse(tree2))  # IE both same bug.

prints

f"{something:'}" 
f'{something:\'}' 
Trees equal:  False
Dumps equal:  True
Unparses equal:  True

Assuming that the tree difference is as unconsequential for compiles as it is for the dumps, the escape looks fine to me. ast.unparse(tree2) == astunparse(

@AlexWaygood AlexWaygood removed needs backport to 3.12 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Nov 24, 2023
@pablogsal
Copy link
Member

pablogsal commented Nov 27, 2023 •
edited
Loading

Assuming that the tree difference is as unconsequential for compiles as it is for the dumps,

Looks like it is:

>>> class A:
...   def __format__(self, *args):
...      print(args)
...      return "A"
...
>>> f"{A():'}"
("'",)
'A'
>>> f"{A():\'}"
("'",)
'A'

and

>>> dis.dis(compile("""f\"{A():\'}\"""", "<string>", "exec"))
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (A)
              6 PRECALL                  0
             10 CALL                     0
             20 LOAD_CONST               0 ("'")
             22 FORMAT_VALUE             4 (with format)
             24 POP_TOP
             26 LOAD_CONST               1 (None)
             28 RETURN_VALUE
>>> dis.dis(compile("""f\"{A():'}\"""", "<string>", "exec"))
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (A)
              6 PRECALL                  0
             10 CALL                     0
             20 LOAD_CONST               0 ("'")
             22 FORMAT_VALUE             4 (with format)
             24 POP_TOP
             26 LOAD_CONST               1 (None)

@15r10nk
Copy link
Contributor Author

15r10nk commented Feb 3, 2024

@pablogsal do you want to fix this issue? I could fix this bug if you want. That would be my first cpython Bugfix.

@pablogsal
Copy link
Member

@pablogsal do you want to fix this issue? I could fix this bug if you want. That would be my first cpython Bugfix.

Yes, let's fix it. Can you make a PR and CC me into it? Let's get your first CPython bug fix in 🚀

@bedevere-app bedevere-app bot mentioned this issue Feb 19, 2024
Merged
15r10nk added a commit to 15r10nk/cpython that referenced this issue Feb 20, 2024
@15r10nk
pythongh-112364: correct unparsing of backslashes and quotes in f-str…
f2f5292 
…ing format_spec
15r10nk added a commit to 15r10nk/cpython that referenced this issue Feb 20, 2024
@15r10nk
pythongh-112364: correct unparsing of backslashes and quotes in f-str…
5290f7f 
…ing format_spec
15r10nk added a commit to 15r10nk/cpython that referenced this issue Feb 20, 2024
@15r10nk
pythongh-112364: correct unparsing of backslashes and quotes in f-str…
ced3764 
…ing format_spec
15r10nk added a commit to 15r10nk/cpython that referenced this issue Feb 21, 2024
@15r10nk
pythongh-112364: correct unparsing of backslashes and quotes in f-str…
c5468f4 
…ing format_spec
pablogsal pushed a commit that referenced this issue Feb 21, 2024
@15r10nk
gh-112364: Correct unparsing of backslashes and quotes in ast.unparse (…
69ab930 
…#115696)
@bedevere-app bedevere-app bot mentioned this issue Feb 21, 2024
Merged
15r10nk added a commit to 15r10nk/cpython that referenced this issue Feb 21, 2024
@15r10nk
[3.12] pythongh-112364: Correct unparsing of backslashes and quotes i…
7c68158 
…n ast.unparse (pythonGH-115696)

(cherry picked from commit 69ab930)

Co-authored-by: Frank Hoffmann <[email protected]>
pablogsal pushed a commit that referenced this issue Feb 21, 2024
@15r10nk
[3.12] gh-112364: Correct unparsing of backslashes and quotes in ast.… (
d2fced7 
#115782)

[3.12] gh-112364: Correct unparsing of backslashes and quotes in ast.unparse (GH-115696)
(cherry picked from commit 69ab930)
@sobolevn sobolevn mentioned this issue Feb 26, 2024
Merged
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this issue Mar 4, 2024
@15r10nk @woodruffw
pythongh-112364: Correct unparsing of backslashes and quotes in ast.u…
c87b5d1 
…nparse (python#115696)
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@15r10nk @diegorusso
pythongh-112364: Correct unparsing of backslashes and quotes in ast.u…
9e8a9c9 
…nparse (python#115696)
LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this issue Jan 22, 2025
@15r10nk @LukasWoodtli
pythongh-112364: Correct unparsing of backslashes and quotes in ast.u…
3bd4380 
…nparse (python#115696)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 only security fixes 3.13 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@pablogsal @terryjreedy @15r10nk @AlexWaygood @Eclips4