From d58495ee70a6062139f2024878d7239ab43c949b Mon Sep 17 00:00:00 2001 From: "C.A.M. Gerlach" Date: Sun, 13 Mar 2022 22:58:53 -0500 Subject: [PATCH 1/3] PEP 685: Update headers & fix reST syntax, links & punctuation --- pep-0685.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/pep-0685.rst b/pep-0685.rst index 41d0d1fe2a7..835e6ce2c75 100644 --- a/pep-0685.rst +++ b/pep-0685.rst @@ -7,7 +7,7 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 08-Mar-2022 -Post-History: 08-Mar-2022 +Post-History: `08-Mar-2022 `__ Abstract @@ -64,10 +64,10 @@ Runs of ``.`` and ``-``, unlike PEP 503, do **not** get collapsed, e.g. ``..`` stays the same. For pip 22, its -"extra normalisation behaviour is quite convoluted and erratic" [pip-erratic]_, +`"extra normalisation behaviour is quite convoluted and erratic" `__, and so its use is not considered. -.. [pip-erratic] https://discuss.python.org/t/what-extras-names-are-treated-as-equal-and-why/7614/10? +.. _pip-erratic: https://discuss.python.org/t/7614/10 Specification @@ -99,7 +99,7 @@ tools SHOULD warn the user. Tools SHOULD warn users when an invalid extra name is read and not use the name to avoid ambiguity. Tools MAY raise an error instead of a warning when reading an -invalid name if they so desire. +invalid name, if they so desire. Backwards Compatibility @@ -108,20 +108,20 @@ Backwards Compatibility Moving to :pep:`503` normalization and :pep:`508` name acceptance, it allows for all preexisting, valid names to continue to be valid. -Based on research looking at a collection of wheels on PyPI [pypi-results]_, +Based on `research looking at a collection of wheels on PyPI `__, the risk of extra name clashes is limited to 73 clashes when considering even invalid names, while *only* looking at valid names leads to only 3 clashes: -1. dev-test: dev_test, dev-test, dev.test -2. dev-lint: dev-lint, dev.lint, dev_lint -3. apache-beam: apache-beam, apache.beam +* ``dev-test``: ``dev_test``, ``dev-test``, ``dev.test`` +* ``dev-lint``: ``dev-lint``, ``dev.lint``, ``dev_lint`` +* ``apache-beam``: ``apache-beam``, ``apache.beam`` By requiring tools writing core metadata to only record the normalized name, the issue of preexisting, invalid extra names should be diminished over time. -.. [pypi-results] https://discuss.python.org/t/pep-685-comparison-of-extra-names-for-optional-distribution-dependencies/14141/17?u=brettcannon +.. _pypi-results: https://discuss.python.org/t/14141/17 Security Implications @@ -162,7 +162,7 @@ Rejected Ideas Using setuptools 60's normalization ----------------------------------- -Initially this PEP proposed following setuptools to try and minimize +Initially, this PEP proposed following setuptools to try to minimize backwards-compatibility issues. But after checking various wheels on PyPI, it became clear that standardizing **all** naming on :pep:`508` and From 09b5431d017db40cc52b8b3e1361fdfc66474dd0 Mon Sep 17 00:00:00 2001 From: "C.A.M. Gerlach" Date: Sun, 13 Mar 2022 23:21:58 -0500 Subject: [PATCH 2/3] PEP 685: Fix grammar & clarity issues, improve phrasing & avoid rep --- pep-0685.rst | 56 ++++++++++++++++++++++++---------------------------- 1 file changed, 26 insertions(+), 30 deletions(-) diff --git a/pep-0685.rst b/pep-0685.rst index 835e6ce2c75..c610b05539c 100644 --- a/pep-0685.rst +++ b/pep-0685.rst @@ -28,13 +28,13 @@ name "must be a valid Python identifier". letter, digit, or any one of ``.``, ``-``, or ``_`` after the initial character. Otherwise, there is no other `PyPA specification `_ -which outlines how extra names should be written or normalization for comparison. +which outlines how extra names should be written or normalized for comparison. Due to the amount of packaging-related code in existence, it is important to evaluate current practices by the community and -standardize on one that doesn't break most code, while being +standardize on one that doesn't break most existing code, while being something tool authors can agree to following. -The issue of there being no standard was brought forward by an +The issue of there being no consistent standard was brought forward by an `initial discussion `__ noting that the extra ``adhoc-ssl`` was not considered equal to the name ``adhoc_ssl`` by pip 22. @@ -47,21 +47,23 @@ Rationale re.sub(r"[-_.]+", "-", name).lower() -This collapses any run of the substitution character down to a single -character, -e.g. ``---`` gets collapsed down to ``-``. -This does **not** produce a valid Python identifier as specified by +This collapses any run of the characters ``-``, ``_`` and ``.`` +down to a single ``-``. +For example, ``---`` ``.`` and ``__`` all get converted to just ``-``. +This does **not** produce a valid Python identifier, per the core metadata 2.2 specification for extra names. -`Setuptools 60 does normalization `__ +`Setuptools 60 performs normalization `__ via:: re.sub(r'[^A-Za-z0-9-.]+', '_', name).lower() -The use of an underscore/``_`` differs from PEP 503's use of a -hyphen/``-``. -Runs of ``.`` and ``-``, unlike PEP 503, do **not** get collapsed, -e.g. ``..`` stays the same. +The use of an underscore/``_`` differs from PEP 503's use of a hyphen/``-``, +and it also normalizes characters outside of those allowed by :pep`508`. +Runs of ``.`` and ``-``, unlike PEP 503, do **not** get normalized to one ``_``, +e.g. ``..`` stays the same. To note, this is inconsistent with this function's +docstring, which *does* specify that all non-alphanumeric characters +(which would include ``-`` and ``.``) are normalized and collapsed. For pip 22, its `"extra normalisation behaviour is quite convoluted and erratic" `__, @@ -96,7 +98,7 @@ name is provided as appropriate for the specified core metadata version. If an older core metadata version is specified and the name would be invalid with newer core metadata versions, tools SHOULD warn the user. -Tools SHOULD warn users when an invalid extra name is read and not use +Tools SHOULD warn users when an invalid extra name is read and SHOULD not use the name to avoid ambiguity. Tools MAY raise an error instead of a warning when reading an invalid name, if they so desire. @@ -105,12 +107,12 @@ invalid name, if they so desire. Backwards Compatibility ======================= -Moving to :pep:`503` normalization and :pep:`508` name acceptance, it +Moving to :pep:`503` normalization and :pep:`508` name acceptance allows for all preexisting, valid names to continue to be valid. Based on `research looking at a collection of wheels on PyPI `__, -the risk of extra name clashes is limited to 73 clashes when considering -even invalid names, +the risk of extra name clashes is limited to 73 instances when considering +all extras names on PyPI, valid or not (not just those within a single package) while *only* looking at valid names leads to only 3 clashes: * ``dev-test``: ``dev_test``, ``dev-test``, ``dev.test`` @@ -118,8 +120,7 @@ while *only* looking at valid names leads to only 3 clashes: * ``apache-beam``: ``apache-beam``, ``apache.beam`` By requiring tools writing core metadata to only record the normalized name, -the issue of preexisting, invalid extra names should be diminished over -time. +the issue of preexisting, invalid extra names should diminish over time. .. _pypi-results: https://discuss.python.org/t/14141/17 @@ -128,7 +129,7 @@ Security Implications ===================== It is possible that for a distribution that has conflicting extra names, a -tool ends up installing distributions that somehow weaken the security +tool ends up installing dependencies that somehow weaken the security of the system. This is only hypothetical and if it were to occur, it would probably be more of a security concern for the distributions @@ -149,7 +150,7 @@ Reference Implementation No reference implementation is provided aside from the code above, but the expectation is the `packaging project`_ will provide a -function in its ``packaging.utils`` that will implement extra name +function in its ``packaging.utils`` module that will implement extra name normalization. It will also implement extra name comparisons appropriately. Finally, if the project ever gains the ability to write out metadata, @@ -162,17 +163,12 @@ Rejected Ideas Using setuptools 60's normalization ----------------------------------- -Initially, this PEP proposed following setuptools to try to minimize -backwards-compatibility issues. -But after checking various wheels on PyPI, +Initially, this PEP proposed using setuptools ``safe_extra()`` for normalization +to try to minimize backwards-compatibility issues. +However, after checking various wheels on PyPI, it became clear that standardizing **all** naming on :pep:`508` and -:pep:`503` semantics was easier and better long-term. - - -Open Issues -=========== - -N/A +:pep:`503` semantics was easier and better long-term, +while causing minimal backwards compatibility issues. Copyright From d3a0a268fac9f257d254d9d8e6579fbe072315b1 Mon Sep 17 00:00:00 2001 From: "C.A.M. Gerlach" Date: Mon, 14 Mar 2022 13:21:26 -0500 Subject: [PATCH 3/3] PEP 685: Restore citations as originally intended --- pep-0685.rst | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/pep-0685.rst b/pep-0685.rst index c610b05539c..020aa33a825 100644 --- a/pep-0685.rst +++ b/pep-0685.rst @@ -66,10 +66,10 @@ docstring, which *does* specify that all non-alphanumeric characters (which would include ``-`` and ``.``) are normalized and collapsed. For pip 22, its -`"extra normalisation behaviour is quite convoluted and erratic" `__, +"extra normalisation behaviour is quite convoluted and erratic" [pip-erratic]_ and so its use is not considered. -.. _pip-erratic: https://discuss.python.org/t/7614/10 +.. [pip-erratic] Tzu-ping Chung on Python Discourse `__, +Based on research looking at a collection of wheels on PyPI [pypi-results]_, the risk of extra name clashes is limited to 73 instances when considering all extras names on PyPI, valid or not (not just those within a single package) while *only* looking at valid names leads to only 3 clashes: @@ -122,7 +122,7 @@ while *only* looking at valid names leads to only 3 clashes: By requiring tools writing core metadata to only record the normalized name, the issue of preexisting, invalid extra names should diminish over time. -.. _pypi-results: https://discuss.python.org/t/14141/17 +.. [pypi-results] Paul Moore on Python Discourse https://discuss.python.org/t/14141/17 Security Implications @@ -171,6 +171,12 @@ it became clear that standardizing **all** naming on :pep:`508` and while causing minimal backwards compatibility issues. +Open Issues +=========== + +N/A + + Copyright =========