Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-40334: PEP 617: New PEG parser for CPython #19503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
Apr 22, 2020
Merged

bpo-40334: PEP 617: New PEG parser for CPython #19503

Changes from 1 commit
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
57f78dd
Run CI in pegen branch as well (#3)
pablogsal Feb 26, 2020
5c0c31f
Merge steps 1-3 and 6 into 'pegen' branch (#8)
gvanrossum Mar 20, 2020
eab06f3
Fall back to a set for done in collect_todo (#9)
lysnikolaou Mar 20, 2020
91427a4
Add Github Actions for pegen tests in Tools/peg_generator (#12)
lysnikolaou Mar 23, 2020
cccfd3d
Allow expressions in yield_expression rule and rename it to annotated…
lysnikolaou Mar 23, 2020
51611b4
Copy .clang-format from pegen project, to guide 'make format-c'
gvanrossum Mar 23, 2020
0c7a382
Expose a C API and a Python-level module for pegen (#11)
lysnikolaou Mar 23, 2020
0cffa81
Run clang-format over peg_parser/pegen.c
gvanrossum Mar 23, 2020
e960c15
Run clang-format over Parser/pegen/pegen.c
gvanrossum Mar 23, 2020
89c2385
Run clang-format over Modules/peg_parser.c
gvanrossum Mar 23, 2020
1d2b107
Ifdef-out token_name() to silence compiler output noise warning
gvanrossum Mar 23, 2020
12f6175
Merge branch 'master' into pegen
gvanrossum Mar 23, 2020
59abf20
Make Ubuntu tests pass; disable Win and Mac tests (#19)
lysnikolaou Mar 23, 2020
14ae717
Rewrite run_parser* functions to always return a mod_ty (#17)
lysnikolaou Mar 24, 2020
bfa7445
Rewrite exported function (#21)
lysnikolaou Mar 24, 2020
4c4917b
Merge remote-tracking branch 'upstream/master' into pegen
lysnikolaou Mar 24, 2020
479bd95
Remove all unnecessary code from Tools/peg_generator; only keep the g…
lysnikolaou Mar 27, 2020
4410737
Support for PEP 614 (#24)
pablogsal Mar 27, 2020
b984cad
Re-enable MacOS test on Github Actions (#26)
lysnikolaou Mar 27, 2020
919a669
Add memoization control (#28)
gvanrossum Mar 27, 2020
55101cb
Fold long lines in metagrammar
gvanrossum Mar 30, 2020
ca3eeca
Update Parser/pegen/parse.c after addition of memoization flag (#28)
lysnikolaou Apr 1, 2020
33b77af
Add a command-line flag and an environment variable to enable pegen (…
lysnikolaou Apr 3, 2020
3aa1dd4
Add missing possible start rules in python.gram (#32)
lysnikolaou Apr 3, 2020
b8ce9a1
Merge remote-tracking branch 'upstream/master' into pegen
pablogsal Apr 3, 2020
f3d8888
Update grammar to account for _Py_keyword changes (#40)
pablogsal Apr 3, 2020
242114a
Merge remote-tracking branch 'upstream/master' into pegen
pablogsal Apr 3, 2020
35bedc2
Add new C API functions PyPegen_*FromFileObject and use them in inter…
lysnikolaou Apr 4, 2020
74736fe
Memoize some rules to avoid exponential time in deep nesting (#39)
pablogsal Apr 4, 2020
4bc4e7c
Run tests with pegen enabled on CI (#41)
lysnikolaou Apr 5, 2020
6ba7869
Improve SyntaxError message when an ERRORTOKEN is returned by the tok…
lysnikolaou Apr 6, 2020
ee7b2cf
Use FILE pointer instead of filename string to pass file to parser (#52)
lysnikolaou Apr 6, 2020
c12c227
Add start_mark variable in loop rules (#51)
lysnikolaou Apr 6, 2020
18743bc
Simplify grammar rules around ASYNC (#53)
gvanrossum Apr 7, 2020
3359dab
Throw SyntaxError when parsing a STRING throws an exception (#54)
lysnikolaou Apr 7, 2020
071df9c
Add use_peg field to sys.flags (#55)
lysnikolaou Apr 7, 2020
b1f0bd8
Improve error reporting and disallow f(**kwargs, *args) (#46)
pablogsal Apr 7, 2020
0627990
Propagate errors from the tokenizer (#57)
pablogsal Apr 8, 2020
f233194
Output more descriptive error messages for named expressions (#56)
lysnikolaou Apr 8, 2020
1dcdc4b
Output more descriptive error messages for invalid assignments (#59)
lysnikolaou Apr 9, 2020
8da51f1
Avoid memory leak in Parser_New (#62)
lysnikolaou Apr 9, 2020
494114e
Fix SEGFAULT in test_fstring (#61)
lysnikolaou Apr 9, 2020
99a8e2f
Provide location info on SyntaxError due to EOLS or EOFS errors (#60)
lysnikolaou Apr 9, 2020
502dfb7
Merge remote-tracking branch 'upstream/master' into pegen
lysnikolaou Apr 10, 2020
e43370e
Ignore encoding cookie when that's necessary (#68)
lysnikolaou Apr 12, 2020
a7f962d
Silence compiler warnings about unused variables on Linux (#70)
lysnikolaou Apr 12, 2020
9c36a0f
Merge remote-tracking branch 'upstream/master' into pegen
lysnikolaou Apr 12, 2020
9952d03
Refactor illegal syntax rules to avoid invalid error messages (#72)
lysnikolaou Apr 13, 2020
b684233
Create a new rule for STRING+ and memoize it (#67)
lysnikolaou Apr 14, 2020
45d6bf7
Fix type of strings rule (#73)
lysnikolaou Apr 14, 2020
a54e89e
Skip one test_traceback test, where pegen is better (#74)
lysnikolaou Apr 15, 2020
d386c70
Skip non-critical test_string_literals test that fails (#75)
lysnikolaou Apr 15, 2020
1169db2
Add script that reproduces all the PEP benchmarks (#71)
lysnikolaou Apr 15, 2020
e764fb2
Merge remote-tracking branch 'upstream/master' into pegen
lysnikolaou Apr 15, 2020
dad5249
Fix calls to _PyInterpreterState_GET after merge
lysnikolaou Apr 15, 2020
6f6354f
Generate specialized error message for invalid string prefixes (#77)
lysnikolaou Apr 15, 2020
b623006
Compress data/xxl.py and decompress on demand (#79)
pablogsal Apr 17, 2020
1a03e3e
Use the new parser by default (#81)
pablogsal Apr 17, 2020
3af6081
Rename pegen functions to start with the _PyPegen_ prefix (#80)
lysnikolaou Apr 17, 2020
e3c95c9
Compile pegen in windows (#82)
pablogsal Apr 17, 2020
d7e4b5c
Remove whitespace to make patchcheck happy (#87)
pablogsal Apr 17, 2020
ffee25f
Rename the peg_parser module to _peg_parser (#86)
pablogsal Apr 17, 2020
b625797
Fix travis errors (#88)
pablogsal Apr 17, 2020
85f7400
Remove the -p command line option and add a new -X oldparser option i…
lysnikolaou Apr 18, 2020
a4c58b5
Disallow trailing comma in from-imports (#97)
lysnikolaou Apr 20, 2020
04062ed
Add blurb
gvanrossum Apr 20, 2020
f1afe08
Use pegen in the symtable and fix symtable test errors (#102)
pablogsal Apr 20, 2020
b8899ec
Merge branch 'master' into pegen
gvanrossum Apr 20, 2020
975baa4
Only optimize __debug__ when ctx==Load (#106)
gvanrossum Apr 21, 2020
298650c
Reproduce current parser's error messages for indentation errors (#107)
lysnikolaou Apr 21, 2020
de6edfb
Add more tokenize errors (#105)
pablogsal Apr 21, 2020
ef0f6fa
Normalize unicode identifiers (#108)
pablogsal Apr 21, 2020
a9d309c
Raise IndentationError on indentation-related errors (#111)
lysnikolaou Apr 21, 2020
871cecb
Copy most fixes from #96 except for the part that uses pegen for comp…
gvanrossum Apr 21, 2020
ecef99b
Temporarily suppress two failures in test_eof.py (one even with old p…
gvanrossum Apr 21, 2020
443afb1
Use col_offset when raising an E_DEDENT error (#116)
lysnikolaou Apr 21, 2020
87460d2
Use pegen in compile and other functions in pythonrun.c (#96)
lysnikolaou Apr 22, 2020
14ab84b
Don't use longjmp and check for the error indicator on every rule (#119)
pablogsal Apr 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix calls to _PyInterpreterState_GET after merge
  • Loading branch information
@lysnikolaou
lysnikolaou committed Apr 15, 2020
commit dad5249bd196664dbc9e3cb1550ba0c2f157fed5
6 changes: 3 additions & 3 deletions Python/pythonrun.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ PyRun_InteractiveOneObjectEx(FILE *fp, PyObject *filename,
PyArena *arena;
const char *ps1 = "", *ps2 = "", *enc = NULL;
int errcode = 0;
int use_peg = _PyInterpreterState_GET_UNSAFE()->config.use_peg;
int use_peg = _PyInterpreterState_GET()->config.use_peg;
_Py_IDENTIFIER(encoding);
_Py_IDENTIFIER(__main__);

Expand Down Expand Up @@ -1030,7 +1030,7 @@ PyRun_StringFlags(const char *str, int start, PyObject *globals,
mod_ty mod;
PyArena *arena;
PyObject *filename;
int use_peg = _PyInterpreterState_GET_UNSAFE()->config.use_peg;
int use_peg = _PyInterpreterState_GET()->config.use_peg;

filename = _PyUnicode_FromId(&PyId_string); /* borrowed */
if (filename == NULL)
Expand Down Expand Up @@ -1061,7 +1061,7 @@ PyRun_FileExFlags(FILE *fp, const char *filename_str, int start, PyObject *globa
mod_ty mod;
PyArena *arena = NULL;
PyObject *filename;
int use_peg = _PyInterpreterState_GET_UNSAFE()->config.use_peg;
int use_peg = _PyInterpreterState_GET()->config.use_peg;

filename = PyUnicode_DecodeFSDefault(filename_str);
if (filename == NULL)
Expand Down