Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Catalog archive contents for single-file input #637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Nov 19, 2021
Merged

Catalog archive contents for single-file input #637

merged 6 commits into from
Nov 19, 2021

Conversation

@wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Nov 18, 2021 •
edited
Loading

This PR adds support for cataloging the contents of a given archive:

# unarchive automatically for input from the "file" scheme
❯ syft file:./hike-gem.tar.gz
...
NAME                               VERSION                         TYPE
hike                               1.2.3                           gem

# unarchive automatically when the "file" scheme is detected
❯ syft ./hike-gem.zip
...
NAME                               VERSION                         TYPE
hike                               1.2.3                           gem

# preserves existing image archive processing
❯ syft ~/oci-images/ubi8.tar
...
NAME                                    VERSION                          TYPE
PyGObject                               3.28.3                           python
PySocks                                 1.6.8                            python
acl                                     2.2.53-1.el8                     rpm
audit-libs                              3.0-0.17.20191104git1c2f876.el8  rpm
basesystem                              11-5.el8                         rpm

Supports zip, tar, and rar archive formats and the following compression formats:

  • brotli (br)
  • bzip2 (bz2)
  • flate (zip)
  • gzip (gz)
  • lz4
  • snappy (sz)
  • xz
  • zstandard (zstd)

Note: this does NOT unarchive contents recursively. That is, if the given archive contains an archive, then the nested archive will be treated as a file and NOT unarchived. Recursive unarchiving should/could be handled under #246.

@wagoodman wagoodman added the enhancement New feature or request label Nov 18, 2021
@wagoodman wagoodman requested a review from a team November 18, 2021 16:02
@wagoodman wagoodman self-assigned this Nov 18, 2021
@github-actions
Copy link

github-actions bot commented Nov 18, 2021 •
edited
Loading

Benchmark Test Results

Benchmark results from the latest changes vs base branch 
name                                                   old time/op    new time/op    delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2          1.68ms ± 1%    1.82ms ± 4%   +7.89%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2        4.28ms ± 9%    4.50ms ± 5%     ~     (p=0.095 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     978µs ± 1%    1060µs ± 1%   +8.34%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                1.17ms ± 1%    1.30ms ± 4%  +11.35%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  986µs ± 1%    1090µs ± 2%  +10.53%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                  13.4ms ± 1%    14.5ms ± 3%   +8.17%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                 1.60ms ± 1%    1.73ms ± 4%   +8.23%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2      1.75µs ± 1%    1.86µs ± 2%   +6.28%  (p=0.008 n=5+5)

name                                                   old alloc/op   new alloc/op   delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2           285kB ± 0%     286kB ± 0%   +0.39%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2        1.29MB ± 0%    1.29MB ± 0%   +0.22%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     211kB ± 0%     211kB ± 0%   +0.15%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                 269kB ± 0%     269kB ± 0%   +0.27%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  234kB ± 0%     234kB ± 0%   +0.10%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                  3.37MB ± 0%    3.36MB ± 0%   -0.27%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                 1.34MB ± 0%    1.35MB ± 0%   +0.03%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2        480B ± 0%      480B ± 0%     ~     (all equal)

name                                                   old allocs/op  new allocs/op  delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2           8.90k ± 0%     8.93k ± 0%   +0.32%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2         38.3k ± 0%     38.5k ± 0%   +0.35%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     5.80k ± 0%     5.82k ± 0%   +0.24%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                 8.51k ± 0%     8.55k ± 0%   +0.49%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  7.09k ± 0%     7.10k ± 0%   +0.20%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                   66.8k ± 0%     66.9k ± 0%   +0.08%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                  11.1k ± 0%     11.1k ± 0%   +0.25%  (p=0.029 n=4+4)
ImagePackageCatalogers/go-module-binary-cataloger-2        11.0 ± 0%      11.0 ± 0%     ~     (all equal)

luhring
luhring reviewed Nov 18, 2021
View reviewed changes
@wagoodman wagoodman requested a review from a team November 19, 2021 12:34
luhring
luhring approved these changes Nov 19, 2021
View reviewed changes
Copy link
Contributor

@luhring luhring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice enhancement! 🙏

@wagoodman wagoodman enabled auto-merge (squash) November 19, 2021 14:15
@wagoodman wagoodman merged commit d76c868 into main Nov 19, 2021
@wagoodman wagoodman deleted the archive-processing branch November 19, 2021 14:16
fengshunli pushed a commit to fengshunli/syft that referenced this pull request Jan 24, 2022
@wagoodman @fengshunli
Catalog archive contents for single-file input (anchore#637)
1eca035 
* add first-level archive processing when input is a file

Signed-off-by: Alex Goodman <[email protected]>

* add license exception for github.com/xi2/xz

Signed-off-by: Alex Goodman <[email protected]>

* always return cleanup function

Signed-off-by: Alex Goodman <[email protected]>

* change source.NewFromFile log entry to warn

Signed-off-by: Alex Goodman <[email protected]>

* ensure file source always has cleanup function

Signed-off-by: Alex Goodman <[email protected]>

* ensure we are always preferring the unarchive cleanup function for source

Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: fsl <[email protected]>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
@wagoodman
Catalog archive contents for single-file input (anchore#637)
336536d 
* add first-level archive processing when input is a file

Signed-off-by: Alex Goodman <[email protected]>

* add license exception for github.com/xi2/xz

Signed-off-by: Alex Goodman <[email protected]>

* always return cleanup function

Signed-off-by: Alex Goodman <[email protected]>

* change source.NewFromFile log entry to warn

Signed-off-by: Alex Goodman <[email protected]>

* ensure file source always has cleanup function

Signed-off-by: Alex Goodman <[email protected]>

* ensure we are always preferring the unarchive cleanup function for source

Signed-off-by: Alex Goodman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@luhring luhring luhring approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@wagoodman wagoodman

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wagoodman @luhring