Thanks to visit codestin.com
Credit goes to github.com

Skip to content

improved adhoc string scanning#386

Merged
domanchi merged 6 commits into
pre-v1-launchfrom
pre-v1-better-adhoc-string-scanning
Jan 13, 2021
Merged

improved adhoc string scanning#386
domanchi merged 6 commits into
pre-v1-launchfrom
pre-v1-better-adhoc-string-scanning

Conversation

@domanchi
Copy link
Copy Markdown
Contributor

Summary

Adhoc string scanning is hella useful. e.g.

$ detect-secrets scan --string 'butactuallynotasecret'
...
Base64HighEntropyString: False

However, sometimes (especially for the HighEntropyString plugins), we want to see why it failed. This is existing behavior in 0.14.3; this PR carries this functionality forward. With this change, it will be:

$ detect-secrets scan --string 'butactuallynotasecret'
...
Base64HighEntropyString: False (3.404)

Reviewer Notes

It was somewhat difficult to surface the non-secret to the scan layer in a generic manner (i.e. without using if plugin in {Base64HighEntropyString}), so I had to be creative with the analyze_line and analyze_string functionality. As a result, for people using the analyze_string function directly (like the test case), they would need to be responsible for filtering out the secret themselves.

Not ideal, but it's the best way I could find.

This PR also introduces another gotcha -- for our DI system, we now leverage the pythonic fashion of duck-typing to determine whether there are enough required parameters to inject into the function. This needed to be done to support cases where the underlying function had default values for their parameters (e.g. HighEntropyStringsPlugin.analyze_line), and as such, we don't need to supply all the parameters that a function needs. However, this also means that if the underlying function raises an uncaught TypeError, it will be silenced by the DI system.

I guess future improvements could be done to distinguish default parameters from non-default parameters in a function declaration, but I didn't want to dive too deep into Python magic internals for this PR.

Copy link
Copy Markdown
Member

@calvinli calvinli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine overall, just have a few comments

Comment thread detect_secrets/core/scan.py Outdated
Comment thread detect_secrets/plugins/high_entropy_strings.py Outdated
Comment thread detect_secrets/plugins/high_entropy_strings.py
Comment thread detect_secrets/core/scan.py Outdated
Comment thread detect_secrets/core/scan.py Outdated
Comment thread detect_secrets/plugins/base.py
Comment thread detect_secrets/util/inject.py Outdated
Copy link
Copy Markdown
Member

@calvinli calvinli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@domanchi domanchi merged commit 481137a into pre-v1-launch Jan 13, 2021
@domanchi domanchi deleted the pre-v1-better-adhoc-string-scanning branch January 13, 2021 23:03
jimmyhlee94 pushed a commit to jimmyhlee94/detect-secrets that referenced this pull request Aug 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants