-
Notifications
You must be signed in to change notification settings - Fork 748
Add experimental cataloger capabilities command #4317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
…apabilities Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Keith Zantow <[email protected]>
internal/capabilities/packages.yaml
Outdated
| default: true | ||
| - name: dependency.depth | ||
| default: | ||
| - direct |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see relationships being returned from
| return []pkg.Package{p}, nil, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you're right, I think the fact that the parser tracked dependencies when parsing threw me. Technically for the dependencies depth description we only need to look at the list of packages, there don't necessarily need to be relationships... so the fact that this returns a single package is the clue.
f41cb60 to
725b0df
Compare
…apabilities Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Keith Zantow <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Will Murphy <[email protected]>
Signed-off-by: Will Murphy <[email protected]>
Signed-off-by: Will Murphy <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to hold this up and overall I think this is fine, especially since there are no changes to the external API, so that's great. In terms of the actual configuration files, I think they are ok -- I'd like them to be split up more (one per parser or cataloger entry), and when a new one is generated it would be nice to have empty defaults for all the fields that need to be manually filled out, but this can always be refined later. Two main thoughts:
I still think we should put capabilities files in the same directory as the cataloger, though it would probably take a little weirdness of a top-level embed (e.g. //go:embed syft/pkg/cataloger/*/*.capability.yaml or similar), which calls back to set the results on the internal package, but it could be done without any external API. Having these in separate locations I'm pretty sure is a large part of the confusion for editing them to me -- if they were in the same directory when people edited the catalogers there would be a significantly higher chance that they are updated properly when making changes that affects these files, especially those that aren't detected by the analysis.
The other holdup I have level of integration when there is still a the dichotomy of detected capability configuration vs. required manual configuration. Including this in a test hook for all PRs for what ends up being a moderately small amount of configuration. In the example cataloger, this is the generated amount of configuration, less than half of what I think is needed to be complete. (And in this particular case, it's wrong because of an oddity that required writing tests differently than expected, but that's probably a bug that should be fixed). By integrating this into the PR process, it gives us the maintenance burden of all the generator code -- almost 10,000 fairly complicated (AST parsing, etc.) LoC to generate on the order of 2,500 LoC -- that gives me pause from a maintenance perspective. I wouldn't want to get rid of this, but I wonder if it was a hint rather than an error, would that reduce the required maintenance for something that is likely to change significantly with 2.0? For example: a user adds a new cataloger without any capabilities, the generator detects what it can and gives the user a new file to edit, it doesn't really matter if it's wrong because the user just manually edits it afterwards. I acknowledge the desire to prevent drift, though. But I foresee this causing some friction for reasons I can't put my finger on -- probably not a large amount of friction, though -- moreso some new cataloger needs to do something unique and generation doesn't work quite right, but because it's all internal we can always change it later.
All that said, this is so very useful for many people, and we can iterate on the details later.
Signed-off-by: Alex Goodman <[email protected]>
…apabilities Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Alex Goodman <[email protected]>
|
I took another shot at moving the cataloger.yaml files, I couldn't easily get it working earlier, but did find a way in the end. It did require introducing an |
This adds the ability, when
SYFT_EXP_CAPABILITIES=true, to use an internalsyft cataloger capscommand to describe cataloger capabilities such as:This is available via an ascii table and JSON output.
Caution
This is an experimental feature and can change without warning and could be removed entirely. Do not depend on this command in production.
The way the capabilities are tracked is described in depth in the internal tooling's readme.
A quick summary is that we use the source code and test observations as a basis for what catalogers exist, how they are configured, and what they output. These things are then used to cross-validate a pseudo-generated
packages/*.yaml(some auto generated items, some manually filled in items) with a set of completion tests (tests that ensure the full universe of things are defined and self-consistent) and then used to drive thecataloger capscommand.Fixes #4155