Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Mar 27, 2023

Each package tracks which files contributed to syft being able to detect the package's existence via the pkg.Package.Locations field. This PR exposes these locations as package-to-file relationships to show that package X is "evident by" file Y. This is made possible with the new evident-by relationship type, which maps to the OTHER SPDX relationship type and depends on a comment value to support decoding effectively.

The current implementation takes all locations from pkg.Package.Location with the "primary" evidence annotation and makes relationships for them. This is nice since there is a single, downstream point in the code that would be responsible for creating these relationships.

In the future application configuration can be introduced which allows for supporting evidence to also be represented with an evident-by relationship type.

This PR also:

  • Fixes a small problem where decoding SPDX relationships of the type OTHER meant for "ownership-by-file-overlap" was interpreted as "dependency-of" instead
  • Fixes Decoding of the syft-json format does not handle files #1534 since this PR adds file nodes to the SBOM which would not be able to be interpreted on multiple encode-decode-encode cycles) which would fail the encode-decode-encode integration test (specifically for the syft json format).

CC: @jedevc @cdupuis

@wagoodman wagoodman added the enhancement New feature or request label Mar 27, 2023
@wagoodman wagoodman requested a review from a team March 27, 2023 17:40
@github-actions
Copy link

github-actions bot commented Mar 27, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz%0A                                                          │ ./.tmp/benchmark-bf92326.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   11.49m ±  1%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             897.6µ ±  1%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.110m ±  2%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   764.1µ ±  4%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       385.7µ ±  1%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   571.0µ ±  1%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   523.7µ ± 18%25%0AImagePackageCatalogers/java-cataloger-2                                     11.19m ±  3%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     91.75µ ±  1%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    613.2µ ±  4%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         92.89µ ±  1%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.199m ±  1%25%0AImagePackageCatalogers/portage-cataloger-2                                  377.6µ ±  1%25%0AImagePackageCatalogers/nix-store-cataloger-2                                273.5µ ±  1%25%0AImagePackageCatalogers/sbom-cataloger-2                                     117.6µ ±  1%25%0AImagePackageCatalogers/binary-cataloger-2                                   199.8µ ±  2%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                             49.61m ±  4%25%0Ageomean                                                                     802.7µ%0A%0A                                                          │ ./.tmp/benchmark-bf92326.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.070Mi ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             140.1Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           982.8Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   179.8Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       98.55Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   169.6Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   178.1Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.777Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.594Ki ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    144.9Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         9.906Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              409.6Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  85.80Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                48.99Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   30.90Ki ± 0%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                             78.44Mi ± 0%25%0Ageomean                                                                     194.5Ki%0A%0A                                                          │ ./.tmp/benchmark-bf92326.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    86.83k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.280k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            15.95k ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    3.797k ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.321k ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    2.989k ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.876k ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.45k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     3.666k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.326k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   1.660k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  886.0 ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     874.0 ± 0%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                              3.001k ± 0%25%0Ageomean                                                                      2.657k

@wagoodman wagoodman self-assigned this Apr 10, 2023
@wagoodman
Copy link
Contributor Author

For interested folks, this PR will be refactored to use location annotations being introduced in #1723 and allow for expressing only primary evidence.

@wagoodman wagoodman marked this pull request as ready for review April 14, 2023 18:55
@wagoodman
Copy link
Contributor Author

JSON schema diff for reviewers:

# ❯ diff schema/json/schema-7.1.3.json schema/json/schema-7.1.4.json
571a572,574
>         },
>         "size": {
>           "type": "integer"
580c583,584
<         "mimeType"
---
>         "mimeType",
>         "size"

@wagoodman wagoodman merged commit 4442285 into main Apr 14, 2023
@wagoodman wagoodman deleted the add-evidence-relationships branch April 14, 2023 19:08
spiffcs added a commit that referenced this pull request Apr 17, 2023
* main: (35 commits)
  Fix kernel cataloger test fixtures (#1742)
  feat: Support scanning license files in golang packages over the network (#1630)
  Add package-to-file location evidence relationships (#1698)
  Add Linux Kernel cataloger (#1694)
  Add annotations for evidence on package locations (#1723)
  add format make target (#1733)
  Update tests to not fail on Mac M1's. (#1730)
  chore(deps): update bootstrap tools to latest versions (#1728)
  Add support for nar files. (#1727)
  add highlevel details about catalogers (#1726)
  chore(deps): bump golang.org/x/net from 0.8.0 to 0.9.0 (#1722)
  chore(deps): update stereoscope to e95d60a265e384df29b7a139f5c5402d6ad72e06 (#1721)
  feat: gradle lockfile support (#1719)
  chore(deps): bump github.com/docker/docker (#1715)
  chore(deps): bump golang.org/x/mod from 0.9.0 to 0.10.0 (#1713)
  chore(deps): bump golang.org/x/term from 0.6.0 to 0.7.0 (#1714)
  chore(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#1716)
  chore(deps): bump peter-evans/create-pull-request from 4 to 5 (#1712)
  chore: update tools-golang to v0.5.0 (#1717)
  Add Nix cataloger (#1696)
  ...

Signed-off-by: Christopher Phillips <[email protected]>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
* add evident-by relationship

Signed-off-by: Alex Goodman <[email protected]>

* wire up evident-by relationship geneation

Signed-off-by: Alex Goodman <[email protected]>

* handle evident-by relationship in spdx formats

Signed-off-by: Alex Goodman <[email protected]>

* fix decoding file info for syft json format

Signed-off-by: Alex Goodman <[email protected]>

* bump json schema to incorporate file size attribute

Signed-off-by: Alex Goodman <[email protected]>

* refactor to create relationships for primary evidence only

Signed-off-by: Alex Goodman <[email protected]>

* fix linting

Signed-off-by: Alex Goodman <[email protected]>

* remove unused 7.0.2 json schema

Signed-off-by: Alex Goodman <[email protected]>

---------

Signed-off-by: Alex Goodman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decoding of the syft-json format does not handle files

3 participants