di · di · Dec 28, 2022 · Dec 28, 2022 · Dec 28, 2022 · Dec 28, 2022
diff --git a/pep-9999.rst b/pep-9999.rst
@@ -0,0 +1,246 @@
+PEP: 9999
+Title: Adding Vulnerability Data to the Simple API for Package Indexes
+Author: Dustin Ingram <[email protected]>
+PEP-Delegate: Donald Stufft <[email protected]>
+Status: Draft
+Type: Standards Track
+Topic: Packaging
+Content-Type: text/x-rst
+Created: 28-Dec-2022
+
+
+Abstract
+========
+
+:pep:`691` defined a JSON form for the "Simple Repository API". This allowed
+clients to more easily query the data that was previously only available in
+HTML, as defined in :pep:`503`.
+
+This proposal adds new fields to the JSON form, which allow it to be used in
+place of PyPI's `JSON API <https://warehouse.pypa.io/api-reference/json.html>`__
+to acquire data about known vulnerabilities for a given project.
+
+The new fields are all part of the data returned from the "project details" URL.
+
+The schema for the vulnerability data and related fields is based on the `Open
+Source Vulnerability Schema <https://ossf.github.io/osv-schema/>`__, provided
+by the `Open Source Vulnerability (OSV) <https://osv.dev/>`__ project of the
+`Open Source Software Security Foundation (OpenSSF) <https://openssf.org/>`__.
+
+
+Rationale
+=========
+
+With the introduction of the JSON form of the simple API in :pep:`691`, the
+simple API offers functionality that is almost as complete as the PyPI JSON API.
+This PEP adds data which were previously only available through the JSON API,
+in order to allow more clients which were previously Warehouse specific to
+support arbitrary standards-compliant indexes.
+
+
+Specification
+=============
+
+This specification defines version 1.2 of the simple repository API. For the
+HTML version of the API, there is no change from version 1.0. For the JSON
+version of the API, the following changes are made:
+
+- The ``api-version`` must specify version 1.2 or later.
+- A new ``vulnerabilities`` key is added at the top level.
+- One new "file information" key, ``vulnerability-ids`` is added to the
+  ``files`` data.
+
+The ``vulnerabilties`` and ``vulnerability-ids`` keys are mandatory.
+
+Vulnerabilities
+---------------
+
+An additional key, ``vulnerabilities`` MUST be present at the top level, in
+addition to the keys previous defined in :pep:`691` and :pep:`700`. This value
+of this key MUST be a mapping from a unique vulnerability identifier to a
+dictionary of fields that describe that vulnerability, for every known
+vulnerability that affects the given project.
+
+An example of the structure for the ``vulnerabilities`` field:
+
+.. code-block:: json
+
+   "vulnerabilities": {
+       "PYSEC-2022-XXX": {
+           "aliases": [
+               "CVE-2022-XXXXX"
+           ],
+           "details": "A long description.",
+           "summary": "A shorter summary.",
+           "fixed_in": [
+               "1.2.3"
+           ],
+           "withdrawn": "2022-06-28T16:39:06Z"
+       }
+   }
+
+Unique vulnerability identifiers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For this field, the key (vulnerability ID) MUST be unique, and MUST take the
+form of the `OSV ID field
+<https://ossf.github.io/osv-schema/#id-modified-fields>`__. If a ``PYSEC-``
+prefixed identifier is available for a given vulnerability, this ID should be
+preferred as the primary vulnerability identifier, and other identifiers should
+be added to the ``aliases`` field.
+
+All of the vulnerability IDs listed in the ``vulnerabilities`` field MUST be
+associated with one or more of the files in the ``files`` key via the
+``vulnerability-ids`` field (see below). If there are no known vulnerabilities
+for a given project, the ``vulnerabilities`` field MUST be an empty mapping.
+
+Dictionary of fields
+~~~~~~~~~~~~~~~~~~~~
+The dictionary of fields that a given vulnerability identifier maps to consists
+of two sets of fields: fields that come from the `Open Source Vulnerabilty
+format (v. 1.3.1) <https://ossf.github.io/osv-schema/>`__, and additional
+field(s) that are specific to this API.
+
+Required fields from the OSV schema
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following fields MUST be included as they are specified in the `Open Source
+Vulnerabilty format (v. 1.3.1) <https://ossf.github.io/osv-schema/>`__:
+
+- ``aliases``: `A list of IDs of the same vulnerability in other databases <https://ossf.github.io/osv-schema/#aliases-field>`__.
+- ``details``: `Additional English textual details about the vulnerability, formatted with CommonMark <https://ossf.github.io/osv-schema/#summary-details-fields>`__.
+- ``summary``: `A one-line, English textual summary of the vulnerability, with no markup <https://ossf.github.io/osv-schema/#summary-details-fields>`__.
+- ``withdrawn`` `The time the entry should be considered to have been withdrawn <https://ossf.github.io/osv-schema/#withdrawn-field>`__.
+
+All other fields in the OSV schema are optional. All fields that return lists
+MUST return empty lists if no value is present. All fields that return strings
+MUST return ``null`` if no value is present.
+
+Additional non-OSV fields
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Additionally, a field ``fixed_in`` MUST be included. This field represents a
+list of versions for which the given vulnerability is considered fixed, and is
+an aggregate of the values of the ``fixed`` field for all events included in
+the ``affected[].ranges[].events`` field of the `OSV specification
+<https://ossf.github.io/osv-schema/#affectedrangesevents-fields>`__ for the
+specific affected version in question.
+
+The purpose of this field is to provide consumers with the information
+necessary to provide a concise upgrade path to mitigate a given vulnerability,
+without requiring API producers to provide the entire set of events that may be
+present in the OSV schema.
+
+An example for how to derive this from the OSV schema format follows:
+
+.. code-block:: python
+
+    fixed_in = [
+        event["fixed"]
+        for affected in schema["affected"]
+        if current_version in affected["versions"]
+        for range in affected["ranges"]
+        for event in range["events"]
+        if "fixed" in event
+    ]
+
+
+Versions listed in the ``fixed_in`` field MUST be present in the ``versions``
+key defined in :pep:`700` after normalizing versions with :pep:`440` (or string
+equivalence for legacy non-:pep:`440` versions). Additionally, versions listed
+in ``fixed_in`` MUST have at least one corresponding file in the ``files`` key
+defined in :pep:`691`.
+
+
+Additional file information
+---------------------------
+
+One new key is added to the ``files`` key:
+
+- ``vulnerability-ids``: This field is mandatory. If a given file has known
+  vulnerabilities associated with it, it MUST contain a list of vulnerability
+  IDs, all of which MUST correspond to keys in the ``vulnerabilties`` top level
+  field. If the file has no known vulnerabilities, this field should be an
+  empty list.
+
+Differences between the legacy JSON API
+=======================================
+
+The following are key differences between this specification and the `Legacy
+PyPI JSON API
+<https://warehouse.pypa.io/api-reference/json.html#known-vulnerabilities>`__.
+
+Removed fields
+--------------
+
+The ``link`` and ``source`` fields have been removed and are no longer present.
+
+
+Differences between the OSV schema
+==================================
+
+The following are key differences between this specification and the `Open
+Source Vulnerability Schema <https://ossf.github.io/osv-schema/>`__.
+
+Python-specific prefix should be preferred
+------------------------------------------
+
+The OSV schema provides support for multiple vulnerability identifier prefixes,
+to provide support for multiple ecosystems. However, this PEP is only focused
+on the Python ecosystem, and thus requires that the primary vulnerability ID
+begin with the ``PYSEC-`` prefix when an ID with that prefix is available. When
+a ``PYSEC-`` prefixed identifier is available, all other identifiers with non
+``PYSEC-`` prefixes should be included in the ``aliases`` field.
+
+Semantic Versioning (SemVer) type should not be used
+----------------------------------------------------
+
+The OSV schema provides support for event types that specify `SemVer
+<https://semver.org/>`__ versions and version ranges. However, because the
+Python packaging ecosystem predates the creation of the SemVer specification,
+many non-SemVer versions exist and are used in practice. As a result, indexes
+should not use the ``SEMVER`` type, and should always use the ``ECOSYSTEM``
+type when specifying versions.
+
+
+FAQ
+===
+
+How should mirrors support this specification?
+----------------------------------------------
+
+Mirrors MAY provide support for this specification. If a mirror is
+redistributing distribution files from a public index like PyPI with
+vulnerability data available, it SHOULD provide corresponding vulnerability
+data as well.
+
+Why not add this data to the HTML API as well?
+----------------------------------------------
+
+See :pep:`700#why-not-add-this-data-to-the-html-api-as-well`.
+
+Does this imply that the HTML API is obsolete?
+----------------------------------------------
+
+See :pep:`700#does-this-imply-that-the-html-api-is-obsolete>`.
+
+Is the simple API replacing the Warehouse JSON and XML-RPC APIs?
+----------------------------------------------------------------
+
+See :pep:`700#is-the-simple-api-replacing-the-warehouse-json-and-xml-rpc-apis`.
+
+Why not require PEP 440 versions?
+---------------------------------
+
+See :pep:`700#why-not-require-pep-440-versions`.
+
+Acknowledgements
+================
+
+This PEP is based significantly on and borrows heavily from :pep:`700`.
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.