Merged
Conversation
With the added automatic sync behavior introduced in git-pkgs#132, we can now experience the latency when the vulnerability data isn't cached. This refactor does a few things: - Add test coverage for `syncVulnerabilitiesForDeps` by extending the signature to accept an injected `source` database instead of having to instantiate inline - Batch-collection of vulnerabilities in groups of 20 to speed up total execution time for large volumes of dependencies - Console progress updater while downloading so the user gets some visual feedback Signed-off-by: Mike Fiedler <[email protected]>
Contributor
Author
|
@andrew This is another bot-guided improvement I came up with, and found some speed and UX gains. With these changes, and on this repository, here's a performance run on my machine: $ hyperfine --min-runs 3 'git pkgs vulns sync --force' './git-pkgs vulns sync --force' --export-json vuln-scan.json
Benchmark 1: git pkgs vulns sync --force
Time (mean ± σ): 13.156 s ± 1.922 s [User: 0.219 s, System: 0.190 s]
Range (min … max): 11.055 s … 14.826 s 3 runs
Benchmark 2: ./git-pkgs vulns sync --force
Time (mean ± σ): 2.061 s ± 0.300 s [User: 0.091 s, System: 0.065 s]
Range (min … max): 1.767 s … 2.367 s 3 runs
Summary
./git-pkgs vulns sync --force ran
6.38 ± 1.32 times faster than git pkgs vulns sync --force |
Contributor
|
Nice, i love a speedup with benchmarks! Do you want to wait on this being merged so it uses #142 |
Contributor
Author
I'm fine shipping this as-is, I don't know when I'll swing back to that one, and that PR will likely refactor this code anyhow. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the added automatic sync behavior introduced in #132, we can now experience the latency when the vulnerability data isn't cached.
This refactor does a few things:
syncVulnerabilitiesForDepsby extending the signature to accept an injectedsourcedatabase instead of having to instantiate inline