Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Dec 8, 2021

Adds the ability to catalog Macho multiarchitecture binaries that are built by the go compiler.

Additionally this PR:

  • raises up the architecture for a given package within a binary in the package metadata section
  • adds the go binary metadata struct to the JSON schema

This relates to goreleaser/goreleaser#2648 (comment)

@wagoodman wagoodman requested a review from a team December 8, 2021 19:19
@wagoodman wagoodman self-assigned this Dec 8, 2021
@wagoodman wagoodman added the enhancement New feature or request label Dec 8, 2021
@wagoodman wagoodman changed the title Add cataloging for universal binaries Add cataloging of universal binaries Dec 8, 2021
@wagoodman wagoodman changed the title Add cataloging of universal binaries Add cataloging of macho multi-architecture binaries Dec 8, 2021
@github-actions
Copy link

github-actions bot commented Dec 8, 2021

Benchmark Test Results

Benchmark results from the latest changes vs base branch
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.41ms ± 1%    1.99ms ± 6%  +40.66%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.57ms ± 1%    4.91ms ± 2%  +37.55%  (p=0.016 n=4+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.04ms ± 1%    1.45ms ± 3%  +38.97%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         838µs ± 1%    1170µs ± 3%  +39.50%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                    1.00ms ± 1%    1.42ms ± 2%  +41.76%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      856µs ± 1%    1247µs ± 4%  +45.71%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      11.2ms ± 1%    15.8ms ±10%  +40.84%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.39ms ± 1%    1.85ms ± 3%  +33.16%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          1.71µs ± 1%    2.27µs ± 4%  +32.86%  (p=0.008 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               296kB ± 0%     297kB ± 0%   +0.14%  (p=0.032 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            1.32MB ± 0%    1.32MB ± 0%   +0.06%  (p=0.032 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     282kB ± 0%     282kB ± 0%   +0.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         222kB ± 0%     222kB ± 0%   +0.18%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     281kB ± 0%     281kB ± 0%   +0.17%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      245kB ± 0%     245kB ± 0%   -0.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.41MB ± 0%    3.41MB ± 0%     ~     (p=0.421 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.36MB ± 0%    1.36MB ± 0%   +0.00%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            560B ± 0%      560B ± 0%     ~     (all equal)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               9.32k ± 0%     9.32k ± 0%     ~     (p=0.444 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             39.7k ± 0%     39.7k ± 0%     ~     (p=0.952 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     9.74k ± 0%     9.74k ± 0%     ~     (p=0.238 n=4+5)
ImagePackageCatalogers/javascript-package-cataloger-2         6.21k ± 0%     6.21k ± 0%     ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     8.99k ± 0%     8.99k ± 0%     ~     (all equal)
ImagePackageCatalogers/rpmdb-cataloger-2                      7.50k ± 0%     7.50k ± 0%     ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       68.8k ± 0%     68.8k ± 0%     ~     (p=0.286 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      11.6k ± 0%     11.6k ± 0%     ~     (p=0.365 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            12.0 ± 0%      12.0 ± 0%     ~     (all equal)

Copy link
Contributor

@luhring luhring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it out and it works like a charm! See comments on code specifics, curious for your thoughts 😃

return []exe{&machoExe{file, e}}, nil
}

if bytes.HasPrefix(data, []byte("\xCA\xFE\xBA\xBE")) || bytes.HasPrefix(data[1:], []byte("\xCA\xFE\xBA\xBF")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be good to add a sense of meaning to this if line in order to motivate these bytes.HasPrefix calls for the reader. If these conditions "mean" something, that'd be great as a factored out function, named for what it's doing. WDYT?

Copy link
Contributor Author

@wagoodman wagoodman Dec 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add some clarity here for the new code added, however, for code that was copied out of the stdlib I have a preference to leave as is (that is, consistent with the other code around it. If we refactor the stdlib code, when we incorporate new stdlib code we need to re-re-factor it, which isn't fun).

Copy link
Contributor

@spiffcs spiffcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - 👍 to @luhring comments, but nothing else to add. Since the code was copied from the standard lib adding comments might make diffing in the future interesting, but I'm all for improvements.

exe.go does have

// This code was copied from the Go std library.
// https://github.com/golang/go/blob/master/src/cmd/go/internal/version/exe.go
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

@wagoodman did add some improvements regarding arch discoverability and stringifying some nasty values.

@wagoodman
Copy link
Contributor Author

Looks like in the future we might be able to get this from exported stdlib code (not released yet though, so not today) golang/go@b37a539

@wagoodman wagoodman merged commit da0b17b into main Dec 8, 2021
@wagoodman wagoodman deleted the add-go-universal-binary-cataloging branch December 8, 2021 21:25
fengshunli pushed a commit to fengshunli/syft that referenced this pull request Jan 24, 2022
* add cataloging within universal binaries

Signed-off-by: Alex Goodman <[email protected]>

* update json test fixtures

Signed-off-by: Alex Goodman <[email protected]>

* add comments + correct 32 bit multi arch magic check

Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: fsl <[email protected]>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
* add cataloging within universal binaries

Signed-off-by: Alex Goodman <[email protected]>

* update json test fixtures

Signed-off-by: Alex Goodman <[email protected]>

* add comments + correct 32 bit multi arch magic check

Signed-off-by: Alex Goodman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants