Codestin Search App

KevinHock · 2020-04-04T18:40:25Z

Adds a script to automate bumping the version, so we don't accidentally cause issue Show helpful error message when plugin not found due to scanning 'new' baseline #269.
🐛 Adds a helpful error message, instead of a raw stack trace, when issue Show helpful error message when plugin not found due to scanning 'new' baseline #269 happens.
Some things I forgot in the initial "Drop Python 2 support" PR (💥 Drop Python 2 support #292)
🎭 Improve the performance of line regexes (Improve the performance of line regexes #244)

Hopefully this avoids needing fixes like ca266b1 Note: I excluded the script from `.pre-commit-config.yaml` to use f-strings We can remove the exclusion when we drop python3.5 support

This fixes issue #269. Previously we showed an unhelpful stack trace.

Missed a few in 5d0d461 Ran `grep -rnw --include=*.py -e 'future' | grep -v three_six | grep -v .tox` this time.

This fixes issue #244. Only check the line for allowlist regexes or --exclude-lines if a secret was found.

OiCMudkips · 2020-04-16T20:26:19Z

+    def _is_excluded_line(self, line):
+        return (
+            any(
+                allowlist_regex.search(line)


Is there a test for this? I thought you needed to wrap the X for X in Y syntax in an array, but I might be wrong.

There is a test 👍

detect-secrets/tests/plugins/high_entropy_strings_test.py

Lines 97 to 133 in ca266b1

@pytest.mark.parametrize(

'content_to_format',

[

# Test inline annotation for allowlisting

"'{secret}' # pragma: allowlist secret",

"'{secret}' # pragma: allowlist secret more text",

"'{secret}' # pragma: allowlist secret",

"'{secret}' // pragma: allowlist secret",

"'{secret}' // pragma: allowlist secret",

"'{secret}' /* pragma: allowlist secret */",

"'{secret}' /* pragma: allowlist secret more text */",

"'{secret}' /* pragma: allowlist secret */",

"'{secret}' ' pragma: allowlist secret",

"'{secret}' ' pragma: allowlist secret",

"'{secret}' -- pragma: allowlist secret",

"'{secret}' -- pragma: allowlist secret",

"'{secret}' ",

"'{secret}' ",

"'{secret}' ",

"'{secret}' ",

# Test old inline annotation for backwards compatibility

"'{secret}' # pragma: whitelist secret",

# Test high entropy exclude regex

'"CanonicalUser": "{secret}"',

# Not a string

'{secret}',

# id occurs before the string, probably a false-positive

'id = "{secret}"',

],

)

def test_ignored_lines(self, content_to_format):

file_content = content_to_format.format(secret=self.secret)

f = mock_file_object(file_content)

results = self.logic.analyze(f, 'does_not_matter')

assert len(results) == 0

>>> san = [True, False, True] >>> fran = {False} >>> any(boo for boo in san) True >>> any(boo for boo in fran) False

KevinHock added 5 commits April 4, 2020 11:10

🐍 Add version bumping script

b3559b4

Hopefully this avoids needing fixes like ca266b1 Note: I excluded the script from `.pre-commit-config.yaml` to use f-strings We can remove the exclusion when we drop python3.5 support

🐍 chmod +x the scripts in scripts/

0b5574a

🐍 Remove unnecessary (object) inheritance

ff6c4a5

🐛 Catch TypeError from from_plugin_classname

eac6919

This fixes issue #269. Previously we showed an unhelpful stack trace.

🐍 Remove all __future__ imports

5ca1d7e

Missed a few in 5d0d461 Ran `grep -rnw --include=*.py -e 'future' | grep -v three_six | grep -v .tox` this time.

KevinHock requested a review from domanchi April 4, 2020 18:40

KevinHock force-pushed the various_small_improvements branch from d5de840 to 5ca1d7e Compare April 4, 2020 18:44

KevinHock added the better exception handling label Apr 4, 2020

🎭 Improve the performance of line regexes

74d3787

This fixes issue #244. Only check the line for allowlist regexes or --exclude-lines if a secret was found.

KevinHock requested a review from OiCMudkips April 7, 2020 05:57

OiCMudkips approved these changes Apr 16, 2020

View reviewed changes

KevinHock merged commit 0e8925d into master Apr 16, 2020

KevinHock deleted the various_small_improvements branch April 16, 2020 20:54

killuazhu pushed a commit to IBM/detect-secrets that referenced this pull request May 28, 2020

Support more db2 patterns (Yelp#293)

01b0d8b

killuazhu pushed a commit to IBM/detect-secrets that referenced this pull request Jul 9, 2020

Support more db2 patterns (Yelp#293)

8c43912

killuazhu pushed a commit to IBM/detect-secrets that referenced this pull request Sep 17, 2020

Support more db2 patterns (Yelp#293)

66b60e9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Various small improvements#293

Various small improvements#293
KevinHock merged 6 commits into
masterfrom
various_small_improvements

KevinHock commented Apr 4, 2020 •

edited

Loading

Uh oh!

OiCMudkips Apr 16, 2020

Uh oh!

KevinHock Apr 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	@pytest.mark.parametrize(
	'content_to_format',
	[
	# Test inline annotation for allowlisting
	"'{secret}' # pragma: allowlist secret",
	"'{secret}' # pragma: allowlist secret more text",
	"'{secret}' # pragma: allowlist secret",
	"'{secret}' // pragma: allowlist secret",
	"'{secret}' // pragma: allowlist secret",
	"'{secret}' /* pragma: allowlist secret */",
	"'{secret}' /* pragma: allowlist secret more text */",
	"'{secret}' /* pragma: allowlist secret */",
	"'{secret}' ' pragma: allowlist secret",
	"'{secret}' ' pragma: allowlist secret",
	"'{secret}' -- pragma: allowlist secret",
	"'{secret}' -- pragma: allowlist secret",
	"'{secret}' <!--pragma: allowlist secret-->",
	"'{secret}' <!-- # pragma: allowlist secret -->",
	"'{secret}' <!-- pragma: allowlist secret -->",
	"'{secret}' <!-- pragma: allowlist secret -->",
	# Test old inline annotation for backwards compatibility
	"'{secret}' # pragma: whitelist secret",
	# Test high entropy exclude regex
	'"CanonicalUser": "{secret}"',
	# Not a string
	'{secret}',
	# id occurs before the string, probably a false-positive
	'id = "{secret}"',
	],
	)
	def test_ignored_lines(self, content_to_format):
	file_content = content_to_format.format(secret=self.secret)
	f = mock_file_object(file_content)

	results = self.logic.analyze(f, 'does_not_matter')

	assert len(results) == 0

Conversation

KevinHock commented Apr 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

OiCMudkips Apr 16, 2020

Choose a reason for hiding this comment

Uh oh!

KevinHock Apr 16, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

KevinHock commented Apr 4, 2020 •

edited

Loading