bpo-40939: Remove documentation for PyParser_* & add porting notes (GH-26855)

encukou · Fidget-Spinner · web-flow · commit 29987f72650b · 2021-06-24T14:57:28.000+02:00
I tried to be relatively thorough and give lots of links.
One reason is that this wasn't deprecated very long; also it seems people running into this tend to not be familiar with similar APIs.

Co-authored-by: Ken Jin &lt;28750310+Fidget-Spinner@users.noreply.github.com&gt;
diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst
@@ -185,42 +185,6 @@ the same library that the Python runtime is using.
       :c:func:`PyMem_RawRealloc`, instead of being allocated by
       :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
 
-
-.. c:function:: struct _node* PyParser_SimpleParseString(const char *str, int start)
-
-   This is a simplified interface to
-   :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving  *filename* set
-   to ``NULL`` and *flags* set to ``0``.
-
-
-.. c:function:: struct _node* PyParser_SimpleParseStringFlags( const char *str, int start, int flags)
-
-   This is a simplified interface to
-   :c:func:`PyParser_SimpleParseStringFlagsFilename` below, leaving  *filename* set
-   to ``NULL``.
-
-
-.. c:function:: struct _node* PyParser_SimpleParseStringFlagsFilename( const char *str, const char *filename, int start, int flags)
-
-   Parse Python source code from *str* using the start token *start* according to
-   the *flags* argument.  The result can be used to create a code object which can
-   be evaluated efficiently. This is useful if a code fragment must be evaluated
-   many times. *filename* is decoded from the :term:`filesystem encoding and
-   error handler`.
-
-
-.. c:function:: struct _node* PyParser_SimpleParseFile(FILE *fp, const char *filename, int start)
-
-   This is a simplified interface to :c:func:`PyParser_SimpleParseFileFlags` below,
-   leaving *flags* set to ``0``.
-
-
-.. c:function:: struct _node* PyParser_SimpleParseFileFlags(FILE *fp, const char *filename, int start, int flags)
-
-   Similar to :c:func:`PyParser_SimpleParseStringFlagsFilename`, but the Python
-   source code is read from *fp* instead of an in-memory string.
-
-
 .. c:function:: PyObject* PyRun_String(const char *str, int start, PyObject *globals, PyObject *locals)
 
    This is a simplified interface to :c:func:`PyRun_StringFlags` below, leaving
diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat
@@ -1791,32 +1791,6 @@ PyObject_TypeCheck:int:::
 PyObject_TypeCheck:PyObject*:o:0:
 PyObject_TypeCheck:PyTypeObject*:type:0:
 
-PyParser_SimpleParseFile:struct _node*:::
-PyParser_SimpleParseFile:FILE*:fp::
-PyParser_SimpleParseFile:const char*:filename::
-PyParser_SimpleParseFile:int:start::
-
-PyParser_SimpleParseFileFlags:struct _node*:::
-PyParser_SimpleParseFileFlags:FILE*:fp::
-PyParser_SimpleParseFileFlags:const char*:filename::
-PyParser_SimpleParseFileFlags:int:start::
-PyParser_SimpleParseFileFlags:int:flags::
-
-PyParser_SimpleParseString:struct _node*:::
-PyParser_SimpleParseString:const char*:str::
-PyParser_SimpleParseString:int:start::
-
-PyParser_SimpleParseStringFlags:struct _node*:::
-PyParser_SimpleParseStringFlags:const char*:str::
-PyParser_SimpleParseStringFlags:int:start::
-PyParser_SimpleParseStringFlags:int:flags::
-
-PyParser_SimpleParseStringFlagsFilename:struct _node*:::
-PyParser_SimpleParseStringFlagsFilename:const char*:str::
-PyParser_SimpleParseStringFlagsFilename:const char*:filename::
-PyParser_SimpleParseStringFlagsFilename:int:start::
-PyParser_SimpleParseStringFlagsFilename:int:flags::
-
 PyRun_AnyFile:int:::
 PyRun_AnyFile:FILE*:fp::
 PyRun_AnyFile:const char*:filename::
diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst
@@ -275,39 +275,8 @@ for more hints.
 
 However sometimes you have to run the embedded Python interpreter in the same
 thread as your rest application and you can't allow the
-:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input.  The one
-solution then is to call :c:func:`PyParser_ParseString` and test for ``e.error``
-equal to ``E_EOF``, which means the input is incomplete.  Here's a sample code
-fragment, untested, inspired by code from Alex Farber::
-
-   #define PY_SSIZE_T_CLEAN
-   #include <Python.h>
-   #include <node.h>
-   #include <errcode.h>
-   #include <grammar.h>
-   #include <parsetok.h>
-   #include <compile.h>
-
-   int testcomplete(char *code)
-     /* code should end in \n */
-     /* return -1 for error, 0 for incomplete, 1 for complete */
-   {
-     node *n;
-     perrdetail e;
-
-     n = PyParser_ParseString(code, &_PyParser_Grammar,
-                              Py_file_input, &e);
-     if (n == NULL) {
-       if (e.error == E_EOF)
-         return 0;
-       return -1;
-     }
-
-     PyNode_Free(n);
-     return 1;
-   }
-
-Another solution is trying to compile the received string with
+:c:func:`PyRun_InteractiveLoop` to stop while waiting for user input.
+A solution is trying to compile the received string with
 :c:func:`Py_CompileString`. If it compiles without errors, try to execute the
 returned code object by calling :c:func:`PyEval_EvalCode`. Otherwise save the
 input for later. If the compilation fails, find out if it's an error or just
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
@@ -1683,9 +1683,9 @@ Removed
   that were only being used by the old parser, including ``node.h``, ``parser.h``,
   ``graminit.h`` and ``grammar.h``.
 
-* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`,
-  :c:func:`PyParser_SimpleParseStringFlagsFilename`,
-  :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile`
+* Removed the Public C API functions ``PyParser_SimpleParseStringFlags``,
+  ``PyParser_SimpleParseStringFlagsFilename``,
+  ``PyParser_SimpleParseFileFlags`` and ``PyNode_Compile``
   that were deprecated in 3.9 due to the switch to the new PEG parser.
 
 * Removed the ``formatter`` module, which was deprecated in Python 3.4.
@@ -1799,6 +1799,41 @@ Changes in the Python API
   also inherits the current builtins.
   (Contributed by Victor Stinner in :issue:`42990`.)
 
+Changes in the C API
+--------------------
+
+* The C API functions ``PyParser_SimpleParseStringFlags``,
+  ``PyParser_SimpleParseStringFlagsFilename``,
+  ``PyParser_SimpleParseFileFlags``, ``PyNode_Compile`` and the type
+  used by these functions, ``struct _node``, were removed due to the switch
+  to the new PEG parser.
+
+  Source should be now be compiled directly to a code object using, for
+  example, :c:func:`Py_CompileString`. The resulting code object can then be
+  evaluated using, for example, :c:func:`PyEval_EvalCode`.
+
+  Specifically:
+
+  * A call to ``PyParser_SimpleParseStringFlags`` followed by
+    ``PyNode_Compile`` can be replaced by calling :c:func:`Py_CompileString`.
+
+  * There is no direct replacement for ``PyParser_SimpleParseFileFlags``.
+    To compile code from a ``FILE *`` argument, you will need to read
+    the file in C and pass the resulting buffer to :c:func:`Py_CompileString`.
+
+  * To compile a file given a ``char *`` filename, explicitly open the file, read
+    it and compile the result. One way to do this is using the :py:mod:`io`
+    module with :c:func:`PyImport_ImportModule`, :c:func:`PyObject_CallMethod`,
+    :c:func:`PyBytes_AsString` and :c:func:`Py_CompileString`,
+    as sketched below. (Declarations and error handling are omitted.) ::
+
+       io_module = Import_ImportModule("io");
+       fileobject = PyObject_CallMethod(io_module, "open", "ss", filename, "rb");
+       source_bytes_object = PyObject_CallMethod(fileobject, "read", "");
+       result = PyObject_CallMethod(fileobject, "close", "");
+       source_buf = PyBytes_AsString(source_bytes_object);
+       code = Py_CompileString(source_buf, filename, Py_file_input);
+
 CPython bytecode changes
 ========================
 
diff --git a/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst b/Misc/NEWS.d/next/C API/2021-06-22-17-00-06.bpo-40939.CGB0I5.rst
@@ -0,0 +1 @@
+Removed documentation for the removed ``PyParser_*`` C API.

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1 @@`
	`1`	+Removed documentation for the removed ``PyParser_*`` C API.