Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit b280248

Browse files
pablogsalpablogsal
authored
bpo-43822: Improve syntax errors for missing commas (GH-25377)
1 parent e692f55 commit b280248

13 files changed

Lines changed: 851 additions & 650 deletions

File tree

‎Doc/library/token-list.inc‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Grammar/Tokens‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ AWAIT
5959
ASYNC
6060
TYPE_IGNORE
6161
TYPE_COMMENT
62+
SOFT_KEYWORD
6263
ERRORTOKEN
6364

6465
# These aren't used by the C tokenizer but are needed for tokenize.py

‎Grammar/python.gram‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ _PyPegen_parse(Parser *p)
77
// Initialize keywords
88
p->keywords = reserved_keywords;
99
p->n_keyword_lists = n_keyword_lists;
10+
p->soft_keywords = soft_keywords;
1011

1112
// Run parser
1213
void *result = NULL;
@@ -459,6 +460,7 @@ expressions[expr_ty]:
459460
| a=expression ',' { _PyAST_Tuple(CHECK(asdl_expr_seq*, _PyPegen_singleton_seq(p, a)), Load, EXTRA) }
460461
| expression
461462
expression[expr_ty] (memo):
463+
| invalid_expression
462464
| a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) }
463465
| disjunction
464466
| lambdef
@@ -778,6 +780,13 @@ invalid_kwarg:
778780
| expression a='=' {
779781
RAISE_SYNTAX_ERROR_KNOWN_LOCATION(
780782
a, "expression cannot contain assignment, perhaps you meant \"==\"?") }
783+
784+
invalid_expression:
785+
# !(NAME STRING) is not matched so we don't show this error with some invalid string prefixes like: kf"dsfsdf"
786+
# Soft keywords need to also be ignored because they can be parsed as NAME NAME
787+
| !(NAME STRING | SOFT_KEYWORD) a=disjunction expression {
788+
RAISE_ERROR_KNOWN_LOCATION(p, PyExc_SyntaxError, a->lineno, a->end_col_offset - 1, "invalid syntax. Perhaps you forgot a comma?") }
789+
781790
invalid_named_expression:
782791
| a=expression ':=' expression {
783792
RAISE_SYNTAX_ERROR_KNOWN_LOCATION(

‎Include/token.h‎

Lines changed: 3 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Lib/test/test_genexps.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@
103103
>>> dict(a = i for i in range(10))
104104
Traceback (most recent call last):
105105
...
106-
SyntaxError: invalid syntax
106+
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?
107107
108108
Verify that parenthesis are required when used as a keyword argument value
109109

‎Lib/test/test_syntax.py‎

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -248,22 +248,36 @@
248248
249249
# Missing commas in literals collections should not
250250
# produce special error messages regarding missing
251-
# parentheses
251+
# parentheses, but about missing commas instead
252252
253253
>>> [1, 2 3]
254254
Traceback (most recent call last):
255-
SyntaxError: invalid syntax
255+
SyntaxError: invalid syntax. Perhaps you forgot a comma?
256256
257257
>>> {1, 2 3}
258258
Traceback (most recent call last):
259-
SyntaxError: invalid syntax
259+
SyntaxError: invalid syntax. Perhaps you forgot a comma?
260260
261261
>>> {1:2, 2:5 3:12}
262262
Traceback (most recent call last):
263-
SyntaxError: invalid syntax
263+
SyntaxError: invalid syntax. Perhaps you forgot a comma?
264264
265265
>>> (1, 2 3)
266266
Traceback (most recent call last):
267+
SyntaxError: invalid syntax. Perhaps you forgot a comma?
268+
269+
# Make sure soft keywords constructs don't raise specialized
270+
# errors regarding missing commas
271+
272+
>>> match x:
273+
... y = 3
274+
Traceback (most recent call last):
275+
SyntaxError: invalid syntax
276+
277+
>>> match x:
278+
... case y:
279+
... 3 $ 3
280+
Traceback (most recent call last):
267281
SyntaxError: invalid syntax
268282
269283
From compiler_complex_args():
@@ -864,7 +878,7 @@
864878
SyntaxError: cannot assign to attribute here. Maybe you meant '==' instead of '='?
865879
866880
Ensure that early = are not matched by the parser as invalid comparisons
867-
>>> f(2, 4, x=34); {1,2 a}
881+
>>> f(2, 4, x=34); 1 $ 2
868882
Traceback (most recent call last):
869883
SyntaxError: invalid syntax
870884

‎Lib/token.py‎

Lines changed: 6 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Misc/NEWS.d/next/Core and Builtins/2021-04-13-02-32-18.bpo-43822.lej0OO.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Improve syntax errors in the parser for missing commas between expressions.
2+
Patch by Pablo Galindo.

0 commit comments

Comments
 (0)